thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayan Moudgill <ma...@bestweb.net>
Subject Re: Streaming
Date Fri, 30 Apr 2010 15:59:13 GMT
Ok, there's a little more documentation about it at:
https://sourceforge.net/apps/mediawiki/cthrift/index.php?title=Asynchrony_%26_Streaming

Still working on figuring out what the right model is for the server.

Mayan Moudgill wrote:
> 
> I've implemented a version of client-side streaming for lists in 
> cthrift. It is _fully_ compatible with the Thrift TBinaryProtocol. It 
> should be possible to adapt these ideas to Thrift.
> 
> It works as follows:
> 1. Split the RPC into a send and a recieve part, so instead of
>   foo()
>  call
>    foo_send()
>    foo_recv()
> 
> 2. For sending a stream, split into the part before the streamed list 
> (including the count), a part that can send portions of the streamed 
> list, and the part after the list. Thus
>   foo_send_stream_start(..., len_of_list)
>   while( !done ) {
>     foo_send_stream_do_stream(..., len_of_chunk, chunk);
>   }
>   foo_send_stream_end(...)
> This idea can be extended to handle multiple streams
> 
> 3. For receiving a stream, the _recv() is split into two functions: the 
> first which returns the count, and the other whcih returns sections of 
> the stream.
>   total = foo_recv_stream_len()
>   n = 0;
>   while( n != total ) {
>     size = foo_recv_stream_data(..., max, res);
>     n += size;
>   }
> Note that max is the maximum chunk size, while size is the size actually 
> read. This can be less than max so that it can handle the case that only 
> part of the stream has been transferred.
> 
> 
> 


Mime
View raw message