thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Chambers <mvchamb...@me.com>
Subject Re: Hi there, I have got a question about the serialization performance using Thrift
Date Sun, 20 Dec 2015 14:35:20 GMT

200k is still a pretty small result.  You should generate the C++ client and hit your server
with it to see if the delay is maybe on the sever side.  Maybe your doing something that is
slowing down the building the result?

If 200k is somehow causing an issue:

Check out:
http://cython.org <http://cython.org/>

What I do for python is generate c++ code, then wrap it with cython. You can stuff like drop
the GIL around each network request, and the extra layer is great place to abstract away all
the connection logic into under the hood thread-local connections.  Exception translation
is tricky but I can show you how to do it if your interested.

-Matt

> On Dec 20, 2015, at 3:18 AM, 刘畅 <liuchang890726@hotmail.com> wrote:
> 
> The background is that, we used to use Pyro4 to do the magic of RPC between our two Python
processes.
> 
> The serializer is cPickle in Pyro4. However, in our new approach, we change one of the
process into C++ using Thrift instead.
> 
> The performance when sending small request/response, Thrift is doing fine.
> 
> But in one of the cases, the size of the response is over 200 KB, and the performance
is dropping rapidly.
> 
> Our implementation detail is listed below:
> 
> Server is using C++, Client is using Python, using TFramedTransport as the transportation
and TThreadedServer as server.
> 
> Unfortunately, our project cannot stand this performance drop between two version. And
I’m here for help.
> 
> I’d like to know whether there is any performance improvement solution in serialization?


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message