thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wilm Schumacher <>
Subject Re: sending large byte objects via thrift
Date Wed, 12 Nov 2014 09:23:03 GMT
So, the memory consumption it is. Thx for the fast reply.

Best wishes,


Am 12.11.2014 um 10:11 schrieb Bryan Pham:
> Hi Will,
> Typically web server are better at handling chunk or stream byte streams
> using disk as a buffer.  Thrift is memory only, so you can't expect to
> scale well for payload of 1GB or 10GB (large file uploads.)  Typically, you
> want the front-end server to handle streaming body and use binary data type
> of maximum 1MB as your back-end infrastructure.
> Best,
> Bryan
> On Wed, Nov 12, 2014 at 1:02 AM, Wilm Schumacher <
>> wrote:
>> Hi,
>> I always read that thrift is not optimized for sending large byte
>> objects (e.g. files) via thrift and that e.g. http should be used instead.
>> However, I want to discuss the use case of connecting a frontend (e.g. a
>> web application) with a backend. This means that I have a handful of
>> connections per thrift server and both client and server are in the same
>> network, perhaps even the same rack. Thus network is only limited by the
>> network card chip
>> In this case I could send a byte object via thrift or create a e.g. http
>> server to recieve and send large obejctes via REST api.
>> But in this use case I do not understand why I shouldn't use thrift.
>> Whether I block a thread for a "http get" or a "thrift send" doesn't
>> make a difference, isn't it?
>> The advantage of the "thrift only" solution would be the simplicity. I
>> would need only one server, and one client per application server. Thus,
>> less code, less errors, less security issues etc..
>> The disadvantage of the "thrift only" solution would be the lack of
>> fancy http features, like compression. But compression wouldn't be a
>> good plan in this use case anyway, because bandwidth isn't the
>> bottleneck in this application of thrift and compression would block a
>> lot CPU.
>> Do I miss something? Is it perhaps the memory consumption of thrift for
>> large messages?
>> Best wishes
>> Wilm

View raw message