thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Meyer <>
Subject Re: multi thread requests handling
Date Thu, 23 Apr 2009 21:13:28 GMT
On Thu, Apr 23, 2009 at 11:38 AM, Debacker <> wrote:

> Thanks for your answer. I should have specified that I would use
> non-blocking asynchronous I/O for the initial read.
> Hence, I can do an async read of max 4096 bytes on 1000 connections, put
> them in a poll, and make a thread wait a completition event of any of the
> 1000 connections.
> Once the completition event is fired, the thread pick one of the ready
> socket, and pass it to a worker thread. The worker thread will begin
> reading
> the beginning of the request, if it is not received completely, we can read
> it synchronously until complete, then make the call to the interface, and
> then write the response. After that, the worker thread give the connection
> back to the master thread, and wait a new job from the master thread. The
> objective here, is to support "sleeping" connections, which don't send any
> request for some a few seconds or minutes. This, in turn, make it possible
> to support connection pools at the client-side, to reduce useless
> connection
> creation/destruction, and the lag of it.
> With the current thread-pooled server, 1000 connections will use 1000
> threads, even if connection are only used for 10% of the time. With my
> system, I would use between 100 and 1000 threads instead.
> Another possibility would be to have one (or few) thread(s) reading
> requests
> from all connections at the same time (using aync I/O and epoll), using one
> state machine per connection. Once the state machine indicates that the
> parsing of the request is complete, we can pass the request to a thread of
> the thread poll, get the response,

I may be mistaken, but I believe this is what the THsHaServer does. The
incoming requests are handled asynchronously using select() and when a
complete request is received (framing is required to detect this) it is
handed off to a thread in the invoker pool.

I believe the THsHaServer is actually very close to what you desire. Also,
if you're running on a system with a modern thread implementation, having a
high number of threads isn't as bad as it once was*. You may be better off
starting with something that's easily available and optimizing if necessary.



and pass the response to another thread
> which will write all response using async I/O concurrently again. That way,
> we do async I/O all the time, and ony the implementation will be blocking.
> But I think it is not possible to implement this using the current
> architecture, that is to say, using the current Protocol classes. And the
> only advantage over my first proposition would be to more efficiency for
> connection which are very slow to transmit a request, which is not that
> important.
> Laurent Debacker.
> On Thu, Apr 23, 2009 at 7:29 PM, David Reiss <> wrote:
> > > I have noticed your non blocking server based on
> > >
> >
> > ,
> > > but you use the framed protocol to read the whole packet completely
> > before
> > > calling the implementation. But this is not needed, as I said in the
> > first
> > > paragraph, we could just read the first chunk of bytes (i.e. 4096) of
> > each
> > > connection. This would required one read buffer per connection, but it
> is
> > > not a problem since the objective would be to reuse the connections.
> >
> > You have to read the entire request before handing off to the processor
> > code.  This is because the processor will attempt to deserialize the
> > entire request.  If it is deserializing from a buffer, it will reach the
> > end and throw an exception, wasting valuable CPU time reading a request
> > that is not complete.  If it is deserializing directly from the socket
> > (or a TBufferedTransport on the socket), it will block until the rest of
> > the message arrives, which makes that thread useless.  Does this make
> > sense?
> >
> > --David
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message