thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debacker <>
Subject Re: multi thread requests handling
Date Thu, 23 Apr 2009 18:38:03 GMT
Thanks for your answer. I should have specified that I would use
non-blocking asynchronous I/O for the initial read.
Hence, I can do an async read of max 4096 bytes on 1000 connections, put
them in a poll, and make a thread wait a completition event of any of the
1000 connections.
Once the completition event is fired, the thread pick one of the ready
socket, and pass it to a worker thread. The worker thread will begin reading
the beginning of the request, if it is not received completely, we can read
it synchronously until complete, then make the call to the interface, and
then write the response. After that, the worker thread give the connection
back to the master thread, and wait a new job from the master thread. The
objective here, is to support "sleeping" connections, which don't send any
request for some a few seconds or minutes. This, in turn, make it possible
to support connection pools at the client-side, to reduce useless connection
creation/destruction, and the lag of it.

With the current thread-pooled server, 1000 connections will use 1000
threads, even if connection are only used for 10% of the time. With my
system, I would use between 100 and 1000 threads instead.

Another possibility would be to have one (or few) thread(s) reading requests
from all connections at the same time (using aync I/O and epoll), using one
state machine per connection. Once the state machine indicates that the
parsing of the request is complete, we can pass the request to a thread of
the thread poll, get the response, and pass the response to another thread
which will write all response using async I/O concurrently again. That way,
we do async I/O all the time, and ony the implementation will be blocking.
But I think it is not possible to implement this using the current
architecture, that is to say, using the current Protocol classes. And the
only advantage over my first proposition would be to more efficiency for
connection which are very slow to transmit a request, which is not that

Laurent Debacker.

On Thu, Apr 23, 2009 at 7:29 PM, David Reiss <> wrote:

> > I have noticed your non blocking server based on
> >
> ,
> > but you use the framed protocol to read the whole packet completely
> before
> > calling the implementation. But this is not needed, as I said in the
> first
> > paragraph, we could just read the first chunk of bytes (i.e. 4096) of
> each
> > connection. This would required one read buffer per connection, but it is
> > not a problem since the objective would be to reuse the connections.
> You have to read the entire request before handing off to the processor
> code.  This is because the processor will attempt to deserialize the
> entire request.  If it is deserializing from a buffer, it will reach the
> end and throw an exception, wasting valuable CPU time reading a request
> that is not complete.  If it is deserializing directly from the socket
> (or a TBufferedTransport on the socket), it will block until the rest of
> the message arrives, which makes that thread useless.  Does this make
> sense?
> --David

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message