thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wilm Schumacher <wilm.schumac...@cawoom.com>
Subject Re: sending large byte objects via thrift
Date Wed, 12 Nov 2014 09:23:03 GMT
So, the memory consumption it is. Thx for the fast reply.

Best wishes,

Wilm

Am 12.11.2014 um 10:11 schrieb Bryan Pham:
> Hi Will,
> 
> Typically web server are better at handling chunk or stream byte streams
> using disk as a buffer.  Thrift is memory only, so you can't expect to
> scale well for payload of 1GB or 10GB (large file uploads.)  Typically, you
> want the front-end server to handle streaming body and use binary data type
> of maximum 1MB as your back-end infrastructure.
> 
> Best,
> Bryan
> 
> On Wed, Nov 12, 2014 at 1:02 AM, Wilm Schumacher <wilm.schumacher@cawoom.com
>> wrote:
> 
>> Hi,
>>
>> I always read that thrift is not optimized for sending large byte
>> objects (e.g. files) via thrift and that e.g. http should be used instead.
>>
>> However, I want to discuss the use case of connecting a frontend (e.g. a
>> web application) with a backend. This means that I have a handful of
>> connections per thrift server and both client and server are in the same
>> network, perhaps even the same rack. Thus network is only limited by the
>> network card chip
>>
>> In this case I could send a byte object via thrift or create a e.g. http
>> server to recieve and send large obejctes via REST api.
>>
>> But in this use case I do not understand why I shouldn't use thrift.
>> Whether I block a thread for a "http get" or a "thrift send" doesn't
>> make a difference, isn't it?
>>
>> The advantage of the "thrift only" solution would be the simplicity. I
>> would need only one server, and one client per application server. Thus,
>> less code, less errors, less security issues etc..
>>
>> The disadvantage of the "thrift only" solution would be the lack of
>> fancy http features, like compression. But compression wouldn't be a
>> good plan in this use case anyway, because bandwidth isn't the
>> bottleneck in this application of thrift and compression would block a
>> lot CPU.
>>
>> Do I miss something? Is it perhaps the memory consumption of thrift for
>> large messages?
>>
>> Best wishes
>>
>> Wilm
>>
> 
> 
> 

Mime
View raw message