thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Craig <ben.cr...@ni.com>
Subject Re: Generic questions
Date Wed, 19 Jun 2013 16:16:27 GMT
Regarding 2), most people end up focusing on the size of messages as a 
benchmark for protocols / serialization methods.  You should evaluate if 
that is the benchmark you care about.  For my projects, I don't really 
care about message size, but I do care about speed.  It is possible that 
"compact" beats "binary" on both of these measures.  It is also entirely 
possible that "binary" is faster to process, but takes up more bandwidth. 
I really don't how the performance stacks up without some set of 
benchmarks.

In C++ with TSocket and TServerSocket, a client does maintain a connection 
to a server.  This has the benefit of only incurring the TCP handshake 
overhead once, but it has the down-side of being more difficult to load 
balance.  However, it shouldn't be too difficult for your application to 
create a Thrift client "on-demand" if the handshake / load balancing 
tradeoff is worth it.



From:   Mark <static.void.dev@gmail.com>
To:     user@thrift.apache.org, 
Date:   06/19/2013 11:01 AM
Subject:        Re: Generic questions



Thanks for the input. So this is what I gathered from your response and 
some investigating. I'm hoping others can chime in with their thoughts.

1) Ill probably want to use either BufferedTransport or FramedTransport. 
Still not quite sure on what should be considered before using one over 
the other. I did some preliminary testing with a ruby server/client and I 
found that BufferedTransport seemed to outperform FramedTransport.

2) BinaryPrototocol is wasteful so other implementations should be 
considered. I'm sure JSON falls into that category too but is a good 
candidate for testing/debugging. So that pretty much leaves Compact as the 
choice of protocol. I also found this to support your argument about 
Binary being wasteful (
http://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html
)

3) The SimpleServer should probably only be used for examples, tests etc. 
That leaves me with the choice of Threaded or NonBlocking but I'm still 
unsure of what should be considered before choosing one over the other. I 
also did some testing with the latest ruby gem and found the 
ThinHttpServer to perform terribly against the other versions. Not quite 
sure why anyone would want to use this.

Do Thrift clients maintain a connection to a server? If so, how does one 
load balance between a pool of servers?

Thanks again

On Jun 19, 2013, at 6:25 AM, Ben Craig <ben.craig@ni.com> wrote:

> Here are the answers for C++.  Often, they apply to the other languages, 

> but I know that some languages (Java) give some of the servers different 

> names.
> 
> 1) The "raw" transports like TSocket do no buffering.  BufferedTransport 

> and FramedTransports are more like transport decorators. 
BufferedTransport 
> adds buffering, but it doesn't change the byte sequence that goes over 
the 
> raw transport.  FramedTransports add buffering, and whenever someone 
calls 
> flush() on the transport, FramedTransport will first write out the size 
of 
> the buffered data, then follow it up with the data.  So FramedTransport 
> changes what bytes are sent / parsed on the raw transport.
> 2) Very little documentation that I've found.  Binary tends to be 
> relatively straightforward, but it isn't doing an awesome job at 
> optimizing for message size.  Compact is a bit trickier, but I haven't 
> looked at it all that much.  I don't know if there are downsides to 
using 
> compact or not.  JSON is almost certainly larger than binary or compact, 

> but it should be easier to debug.
> 3) In C++, the simple server only permits one connection.  The threaded 
> server allocates one thread per connection.  The nonblocking server is 
> strange, and I haven't really used it yet.  I know that it requires a 
> framed transport, and requires you to use something socket based because 

> it has a select loop in it.  It claims to be high performance.



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message