thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonas Ruesch <jrue...@jruesch.ch>
Subject Re: TZLibTransport for CSharp?
Date Sun, 01 May 2016 19:37:45 GMT
Hi Jens,

To assess the performance (with respect to time) we tested only the
existing C++ TZlibTransport implementation (client and server both C++,
running on the same machine, communicating via tcp). In our current
productive setup one side is always C++, the other C# (or C++).

Cheers,
Jonas


On 29 April 2016 at 19:48, Jens Geyer <jensgeyer@hotmail.com> wrote:

> Hi Jonas,
>
> short question: What is your (non-C#) counterpart that you use to test
> against?
>
> Thanks,
> JensG
>
>
>
> -----Urspr√ľngliche Nachricht----- From: Randy Abernethy
> Sent: Friday, April 29, 2016 5:57 PM
>
> To: user@thrift.apache.org
> Subject: Re: TZLibTransport for CSharp?
>
> Hey Jonas,
>
> Thanks for the detailed write up! Will provide great guidance for anyone
> interested in implementing TZlibTransport for C#. My experience concurs
> with yours, using ZLib for thrift file compression can be useful but it is
> not generally an uptick for RPC. Perhaps in that context lack of flush is
> acceptable.
>
> -Randy
>
> On Fri, Apr 29, 2016 at 7:16 AM, Jonas Ruesch <jruesch@jruesch.ch> wrote:
>
> Hi Randy, Jens,
>>
>> Following your suggestions and the existing Java implementation, we had a
>> closer look at a possible C# implementation. Below I add a few technical
>> notes on what we found. So far we didn't pursue any of the options
>> further,
>> because for our specific application we found that in practice the
>> uncompressed binary format outperformed the compressed communication in
>> terms of time per call (tested with C++ client & server).
>>
>> Technical notes on possible TZlibTransport implementations in C#/.Net
>> (credit: Mario G.):
>>
>> Dot net supports by default a compression format which is compatible to
>> ZLib. It is little effort to create some wrappers to implement the
>> TZlibTransport for DotNet. However, the issue is that Thrift is forcing to
>> send data by flushing the stream, but the DotNet compressions streams do
>> not support flushing. This means that data is only being sent if there is
>> enough data available to compress the next block, or if the stream is
>> closed.
>>
>> Thrift only closes the sending stream after the response has been
>> received.
>> But as the data has never been sent, data is never received too.
>>
>> To solve this issue the compression stream can be closed instead of
>> flushed. This works as long as it is guaranteed that thrift never sends
>> any
>> additional data after calling flush. We didn't investigate if this is
>> guaranteed (todo).
>>
>> If the above condition is not guaranteed, flush support could be
>> implemented by using the DotNetZip library which supports the sync_flush
>> mode (http://www.bolet.org/~pornin/deflate-flush.html). This mode is only
>> supported on a very low level, thus a lot of handling has to be
>> implemented
>> manually.
>>
>> (Sorry, this answer might be better suited for the developer mailing list,
>> but I kept it in this thread for better context.)
>>
>> Best,
>> Jonas
>>
>>
>>
>>
>> On 15 February 2016 at 19:30, Randy Abernethy <ra@apache.org> wrote:
>>
>> > Hey Jonas,
>> >
>> > I think .Net has support for deflate built in, so there's not need for a
>> > third party lib. More info here:
>> >
>> >
>>
>> https://msdn.microsoft.com/en-us/library/system.io.compression.deflatestream(v=vs.110).aspx
>> >
>> > Should be pretty easy to use the Java or C++ TZlibTrans as a model and
>> get
>> > the same going in C#.
>> >
>> > Best,
>> > Randy
>> >
>> >
>> > On Sun, Feb 14, 2016 at 11:59 PM, Jonas Ruesch <jruesch@jruesch.ch>
>> wrote:
>> >
>> > > Hi Jens, Randy,
>> > >
>> > > Thanks for confirming.
>> > > Any preference or previous experiences with a zlib-compatible zip
>> library
>> > > for c#? That one looks promising: http://dotnetzip.codeplex.com/
>> > >
>> > > Cheers,
>> > > Jonas
>> > >
>> > >
>> > > On 13 February 2016 at 00:36, Jens Geyer <jensgeyer@hotmail.com>
>> wrote:
>> > >
>> > > > Hi Jonas,
>> > > >
>> > > > patches and pull requests are welcome.
>> > > >
>> > > > Because the documentation states 'zlib transport not available for
>> > java'
>> > > >> (in the meantime it seems to be available though) I had hopes
that
>> > there
>> > > >> is
>> > > >> a C# implementation.
>> > > >>
>> > > >
>> > > > That's the problem with logic. It sometimes hits you where you >
>
>> > expect
>> > it
>> > > > the least. Just kidding ... ;-) As I said on SO, some things do not
>> > exist
>> > > > simply because nobody needed them so far. But that does not mean it
>> > can't
>> > > > or shouldn't be added if someone does need it. Especially TZlib is
a
>> > good
>> > > > thing to have.
>> > > >
>> > > > Looking forward to your contribution. If you need help with it, just
>> > ask,
>> > > > and someone will answer.
>> > > >
>> > > > Have fun,
>> > > > JensG
>> > > >
>> > > >
>> > > > -----Urspr√ľngliche Nachricht----- From: Randy Abernethy
>> > > > Sent: Friday, February 12, 2016 3:48 PM
>> > > > To: user@thrift.apache.org
>> > > > Subject: Re: TZLibTransport for CSharp?
>> > > >
>> > > >
>> > > > Hi Jonas,
>> > > >
>> > > > The current master does not have a TZlibTransport for C#. Would be
a
>> > > great
>> > > > add though! Patches always welcome.
>> > > >
>> > > > Best,
>> > > > Randy
>> > > >
>> > > > On Fri, Feb 12, 2016 at 2:46 AM, Jonas Ruesch <jruesch@jruesch.ch>
>> > > wrote:
>> > > >
>> > > > Hi,
>> > > >>
>> > > >> Is there an implementation of the zlib Transport in C# (Thrift
>> > 0.9.3)? I
>> > > >> looked for it in the lib/csharp source code and believe to have
>> > searched
>> > > >> the web extensively to no avail.
>> > > >>
>> > > >> Because the documentation states 'zlib transport not available
for
>> > java'
>> > > >> (in the meantime it seems to be available though) I had hopes
that
>> > there
>> > > >> is
>> > > >> a C# implementation.
>> > > >>
>> > > >> Sorry if this question was asked before (couldn't find any traces).
>> > > >>
>> > > >> Many thanks for your hints.
>> > > >>
>> > > >>
>> > > >
>> > >
>> >
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message