thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Reiss <dre...@facebook.com>
Subject Re: Transmission of big data
Date Mon, 26 Apr 2010 19:37:17 GMT
In addition to all of Bryan's points, keep in mind that Thrift was designed
to deal with structured data, so it makes some unnecessary memory copies
that could be eliminated in an implementation meant more for binary
blobs.  If you are trying to maximize performance, this could get in
your way.

--David

Bryan Duxbury wrote:
> You can use Thrift for transferring big stuff up to a point. The limit you
> run into is that whole Thrift struct you create has to fit into memory, so
> if you have 10GB of stuff to transfer and only 1GB of memory, you'd have to
> do more than one RPC.
> 
> Also be aware that even if you have a lot of memory, I don't think that we
> can encode binary fields bigger than 2**32 bytes in length, since all our
> protocols specify an int as the array size.
> 
> On Sat, Apr 24, 2010 at 12:49 AM, CARMELO VELARDO <carmelo.velardo@gmail.com
>> wrote:
> 
>> Hi to all,
>>
>> I just subscribed to the newsletter.
>> I hope not to sound noob but...
>>
>> does thrift is intended only for small data exchanging,
>> or also big data (images, documents) can be transferred
>> from client to server?
>>
>> Thank you,
>>
>> Carmelo
>>

Mime
View raw message