thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jens Geyer" <jensge...@hotmail.com>
Subject Re: TZLibTransport for CSharp?
Date Fri, 29 Apr 2016 17:48:29 GMT
Hi Jonas,

short question: What is your (non-C#) counterpart that you use to test 
against?

Thanks,
JensG



-----Urspr√ľngliche Nachricht----- 
From: Randy Abernethy
Sent: Friday, April 29, 2016 5:57 PM
To: user@thrift.apache.org
Subject: Re: TZLibTransport for CSharp?

Hey Jonas,

Thanks for the detailed write up! Will provide great guidance for anyone
interested in implementing TZlibTransport for C#. My experience concurs
with yours, using ZLib for thrift file compression can be useful but it is
not generally an uptick for RPC. Perhaps in that context lack of flush is
acceptable.

-Randy

On Fri, Apr 29, 2016 at 7:16 AM, Jonas Ruesch <jruesch@jruesch.ch> wrote:

> Hi Randy, Jens,
>
> Following your suggestions and the existing Java implementation, we had a
> closer look at a possible C# implementation. Below I add a few technical
> notes on what we found. So far we didn't pursue any of the options 
> further,
> because for our specific application we found that in practice the
> uncompressed binary format outperformed the compressed communication in
> terms of time per call (tested with C++ client & server).
>
> Technical notes on possible TZlibTransport implementations in C#/.Net
> (credit: Mario G.):
>
> Dot net supports by default a compression format which is compatible to
> ZLib. It is little effort to create some wrappers to implement the
> TZlibTransport for DotNet. However, the issue is that Thrift is forcing to
> send data by flushing the stream, but the DotNet compressions streams do
> not support flushing. This means that data is only being sent if there is
> enough data available to compress the next block, or if the stream is
> closed.
>
> Thrift only closes the sending stream after the response has been 
> received.
> But as the data has never been sent, data is never received too.
>
> To solve this issue the compression stream can be closed instead of
> flushed. This works as long as it is guaranteed that thrift never sends 
> any
> additional data after calling flush. We didn't investigate if this is
> guaranteed (todo).
>
> If the above condition is not guaranteed, flush support could be
> implemented by using the DotNetZip library which supports the sync_flush
> mode (http://www.bolet.org/~pornin/deflate-flush.html). This mode is only
> supported on a very low level, thus a lot of handling has to be 
> implemented
> manually.
>
> (Sorry, this answer might be better suited for the developer mailing list,
> but I kept it in this thread for better context.)
>
> Best,
> Jonas
>
>
>
>
> On 15 February 2016 at 19:30, Randy Abernethy <ra@apache.org> wrote:
>
> > Hey Jonas,
> >
> > I think .Net has support for deflate built in, so there's not need for a
> > third party lib. More info here:
> >
> >
> https://msdn.microsoft.com/en-us/library/system.io.compression.deflatestream(v=vs.110).aspx
> >
> > Should be pretty easy to use the Java or C++ TZlibTrans as a model and
> get
> > the same going in C#.
> >
> > Best,
> > Randy
> >
> >
> > On Sun, Feb 14, 2016 at 11:59 PM, Jonas Ruesch <jruesch@jruesch.ch>
> wrote:
> >
> > > Hi Jens, Randy,
> > >
> > > Thanks for confirming.
> > > Any preference or previous experiences with a zlib-compatible zip
> library
> > > for c#? That one looks promising: http://dotnetzip.codeplex.com/
> > >
> > > Cheers,
> > > Jonas
> > >
> > >
> > > On 13 February 2016 at 00:36, Jens Geyer <jensgeyer@hotmail.com>
> wrote:
> > >
> > > > Hi Jonas,
> > > >
> > > > patches and pull requests are welcome.
> > > >
> > > > Because the documentation states 'zlib transport not available for
> > java'
> > > >> (in the meantime it seems to be available though) I had hopes that
> > there
> > > >> is
> > > >> a C# implementation.
> > > >>
> > > >
> > > > That's the problem with logic. It sometimes hits you where you 
> > > > expect
> > it
> > > > the least. Just kidding ... ;-) As I said on SO, some things do not
> > exist
> > > > simply because nobody needed them so far. But that does not mean it
> > can't
> > > > or shouldn't be added if someone does need it. Especially TZlib is a
> > good
> > > > thing to have.
> > > >
> > > > Looking forward to your contribution. If you need help with it, just
> > ask,
> > > > and someone will answer.
> > > >
> > > > Have fun,
> > > > JensG
> > > >
> > > >
> > > > -----Urspr√ľngliche Nachricht----- From: Randy Abernethy
> > > > Sent: Friday, February 12, 2016 3:48 PM
> > > > To: user@thrift.apache.org
> > > > Subject: Re: TZLibTransport for CSharp?
> > > >
> > > >
> > > > Hi Jonas,
> > > >
> > > > The current master does not have a TZlibTransport for C#. Would be a
> > > great
> > > > add though! Patches always welcome.
> > > >
> > > > Best,
> > > > Randy
> > > >
> > > > On Fri, Feb 12, 2016 at 2:46 AM, Jonas Ruesch <jruesch@jruesch.ch>
> > > wrote:
> > > >
> > > > Hi,
> > > >>
> > > >> Is there an implementation of the zlib Transport in C# (Thrift
> > 0.9.3)? I
> > > >> looked for it in the lib/csharp source code and believe to have
> > searched
> > > >> the web extensively to no avail.
> > > >>
> > > >> Because the documentation states 'zlib transport not available for
> > java'
> > > >> (in the meantime it seems to be available though) I had hopes that
> > there
> > > >> is
> > > >> a C# implementation.
> > > >>
> > > >> Sorry if this question was asked before (couldn't find any traces).
> > > >>
> > > >> Many thanks for your hints.
> > > >>
> > > >>
> > > >
> > >
> >
> 


Mime
View raw message