thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Pham <br...@cloudtenna.com>
Subject Re: sending large byte objects via thrift
Date Wed, 12 Nov 2014 09:11:20 GMT
Hi Will,

Typically web server are better at handling chunk or stream byte streams
using disk as a buffer.  Thrift is memory only, so you can't expect to
scale well for payload of 1GB or 10GB (large file uploads.)  Typically, you
want the front-end server to handle streaming body and use binary data type
of maximum 1MB as your back-end infrastructure.

Best,
Bryan

On Wed, Nov 12, 2014 at 1:02 AM, Wilm Schumacher <wilm.schumacher@cawoom.com
> wrote:

> Hi,
>
> I always read that thrift is not optimized for sending large byte
> objects (e.g. files) via thrift and that e.g. http should be used instead.
>
> However, I want to discuss the use case of connecting a frontend (e.g. a
> web application) with a backend. This means that I have a handful of
> connections per thrift server and both client and server are in the same
> network, perhaps even the same rack. Thus network is only limited by the
> network card chip
>
> In this case I could send a byte object via thrift or create a e.g. http
> server to recieve and send large obejctes via REST api.
>
> But in this use case I do not understand why I shouldn't use thrift.
> Whether I block a thread for a "http get" or a "thrift send" doesn't
> make a difference, isn't it?
>
> The advantage of the "thrift only" solution would be the simplicity. I
> would need only one server, and one client per application server. Thus,
> less code, less errors, less security issues etc..
>
> The disadvantage of the "thrift only" solution would be the lack of
> fancy http features, like compression. But compression wouldn't be a
> good plan in this use case anyway, because bandwidth isn't the
> bottleneck in this application of thrift and compression would block a
> lot CPU.
>
> Do I miss something? Is it perhaps the memory consumption of thrift for
> large messages?
>
> Best wishes
>
> Wilm
>



-- 
Best,
Bryan Pham
Founder, CloudTenna Inc.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message