thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Meyer <joel.me...@gmail.com>
Subject Re: Java Server Does Not Close Sockets immediately for oneway methods
Date Thu, 11 Feb 2010 18:02:40 GMT
Utku, your strategy of using a ThreadPoolExecutor to handle the message
processing (as opposed to the message parsing, which is done by the Thrift
server) seems like the right solution. Depending on your reliability
requirements and the size of your one-way method invocations, I imagine you
could emit UDP packets containing serialized thrift objects and it may be
faster and require fewer connections (just one to read and enqueue incoming
messages).

If you're on *nix you can increase the number of file handles available to
the Java process by calling "ulimit -n 131000" in the Java server start up
script. On Centos/Redhad the max number that can be passed to ulimit is
controlled in /etc/security/limits.conf.

On Thu, Feb 11, 2010 at 9:49 AM, Utku Can Topçu <utku@topcu.gen.tr> wrote:

> I'm sorry to post once more again,
> The connection is apparently useless, and keeping it open causes "too many
> open files" eventually.
>
> On Thu, Feb 11, 2010 at 7:46 PM, Utku Can Topçu <utku@topcu.gen.tr> wrote:
>
> > David, Thank you for the quick response
> >
> > Since the method is oneway as defined by Thrift IDL, should we still
> insist
> > on keeping the connection alive until the method finishes? Because of the
> > "asynchronous call" paradigm, the assumption is the fact that the client
> > would not care about the execution, result etc... of the related method.
> >
> > Best,
> > Utku
> >
> >
> >
> > On Thu, Feb 11, 2010 at 7:42 PM, David Reiss <dreiss@facebook.com>
> wrote:
> >
> >> That sounds right.  The client closes the socket, the server is still
> >> processing the method, and it closes its side as soon as it finishes.
> >>
> >> Utku Can Topçu wrote:
> >> > Hello David,
> >> >
> >> > I think there's a point missing. Even though the client (PHP in our
> >> case)
> >> > issues a "$transport->close()" the socket connection on the Server
> Side
> >> is
> >> > closed just after the execution of the oneway method.
> >> >
> >> > Best,
> >> > Utku
> >> >
> >> > On Thu, Feb 11, 2010 at 7:28 PM, David Reiss <dreiss@facebook.com>
> >> wrote:
> >> >
> >> >> We don't close the sockets because you are allowed to send multiple
> >> >> requests
> >> >> over the same connection.
> >> >>
> >> >> Requests received over the same connection are processed serially,
in
> >> >> order.
> >> >>
> >> >> Does that answer your question?
> >> >>
> >> >> Utku Can Topçu wrote:
> >> >>> While experimenting with the oneway (asynchronous) service calls
to
> a
> >> >> Java
> >> >>> Service, we've faced a serious connection overload.
> >> >>> Currently we're using TBufferedTransport together with
> >> TThreadPoolServer
> >> >> on
> >> >>> the Server side.
> >> >>> Diving deep into the Java Thrift Libraries, auto-generated Java
> >> classes
> >> >> and
> >> >>> debugging the code, we've seen that the oneway void methods on
the
> >> Java
> >> >>> Server side does not close the "TSocket"s that are related to the
> >> methods
> >> >> in
> >> >>> question.
> >> >>>
> >> >>> Is this a design decision? Should we treat this as a bug?
> >> >>>
> >> >>> Our current solution to this problem is employing a
> ThreadPoolExecutor
> >> >> for
> >> >>> each oneway method, we immediately transform the asynchronous
> requests
> >> >> into
> >> >>> Runnables and place them in the thread pool and return from the
> >> method.
> >> >>>
> >> >>> I think a similar approach should be implemented in the Thrift
Java
> >> >>> libraries.
> >> >>>
> >> >>> What do you think on this issue?
> >> >>>
> >> >>> Best Regards,
> >> >>> Utku
> >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message