thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Chambers <>
Subject Re: Dealing with out of sequence responses
Date Fri, 30 Oct 2015 14:20:16 GMT

I’ve found it that if a timeout occurs on the client then its a toss up on if the communication
stack will recover. So, if anything bad happens I tear the whole thing down and bring it back
up, usually pointing at a different server in the cluster.  If the call is idempotent I automatically
retry the same API call on the new server.  So most of the time people don’t even know.
Most timeouts occur when we’re doing rolling restarts of the servers.  If I run out of servers
I just reset the list and keep trying until they come back.

> On Oct 30, 2015, at 10:10 AM, Tyler Treat <> wrote:
> I guess my main question then is what the best practice is when dealing
> with timeouts. It sounds like if a request times out, you should disconnect
> the client and reconnect, which is an unfortunate amount of overhead for
> what is potentially a transient error.
> Thanks,
> Tyler
> On Fri, Oct 30, 2015 at 2:54 AM Jens Geyer <> wrote:
>> The single most important thing that comes to my mind is this: What if the
>> client is aware of this and *wants* both responses?
>> If the framework automatically and implicitly drops older requests or
>> refuses to deal with responses coming in a different order, how do you
>> handle that? But if that logic is one layer above Thrift, it is much easier
>> implement whatever model you need.
>> There are situations for each model, IMHO. If we plan to do anything here,
>> we should add comfort, but keep all possibilities.
>> Have fun,
>> JensG
>> ________________________________
>> Von: Tyler Treat
>> Gesendet: 30.10.2015 01:06
>> An:
>> Betreff: Dealing with out of sequence responses
>> Hi,
>> I'm curious about Thrift's design around out of sequence responses.
>> Consider the sequence of events in this contrived example:
>> Client calls ping()
>> Server receives ping() and sleeps a few seconds
>> Client ping() times out
>> Client calls add()
>> Server finishes sleeping and responds to ping()
>> Client gets out of sequence response error (received ping() response when
>> expecting add() response)
>> It seems like what Thrift should do is drop responses with previous
>> sequence ids? I.e. if ping() sequence id is 1 and it times out and I call
>> add() which expects sequence id 2, I should drop the response with id 1.
>> Am I thinking about this correctly?
>> Thanks,
>> Tyler

View raw message