thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nobuaki Sukegawa <nsu...@gmail.com>
Subject Re: Hi there, I have got a question about the serialization performance using Thrift
Date Sun, 20 Dec 2015 11:03:41 GMT
Which protocol do you use ?
During the observed performance hit, memory is sufficient and CPU is fully
utilized ?

If you're using binary protocol, TBinaryProtocolAccelerated class is an
option.
It uses C extension and in my past local test, was a lot faster than pure
python counter part.
Unfortunately, there's a known problem with large collections in latest
0.9.3 so you may need to downgrade to 0.9.2 with this one.

There are also a few generator switches to change the run-time behavior
like -gen py:dynamic etc.
Some of them claim that they dramatically reduce memory consumption.

On Sun, Dec 20, 2015 at 5:19 PM 刘畅 <liuchang890726@hotmail.com> wrote:

> The background is that, we used to use Pyro4 to do the magic of RPC
> between our two Python processes.
>
> The serializer is cPickle in Pyro4. However, in our new approach, we
> change one of the process into C++ using Thrift instead.
>
> The performance when sending small request/response, Thrift is doing fine.
>
> But in one of the cases, the size of the response is over 200 KB, and the
> performance is dropping rapidly.
>
> Our implementation detail is listed below:
>
> Server is using C++, Client is using Python, using TFramedTransport as the
> transportation and TThreadedServer as server.
>
> Unfortunately, our project cannot stand this performance drop between two
> version. And I’m here for help.
>
> I’d like to know whether there is any performance improvement solution in
> serialization?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message