thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henrique Mendonça <henri...@apache.org>
Subject Re: Why thrift serialization is so much slow-slow?
Date Tue, 08 Jan 2013 23:34:17 GMT
Hi Vincent,

Do you have any reason to use test_json(rawdata) instead of test_json(data)
?
It looks like the object is a lot smaller on rawdata...
Anyway you're trying to compare a built-in serialisation function with the
binary implementation, and in this case I would expect some lost of
performance.
You can also try to use the compact protocol, as it's also supposed to be a
little faster, and/or the new json protocol. With you have a benchmark on
that, please report back to us.

Regards,
Henrique

On 6 January 2013 09:43, Vincent <phostu@gmail.com> wrote:

> Hi, all
>
> I'm using thrift in python, I found serialize structure-data in
> thrift-python is very slow.
>
> I wrote a serialization test on thrift and json,
>
> *testing thrift defination:*
> https://gist.github.com/4465825
> https://gist.github.com/4465826
>
> *python testing code:*
> https://gist.github.com/4465830
>
> *testing data:*
> https://gist.github.com/4465834
>
> *testing results:*
> https://gist.github.com/4465853
>
>
> testing results:
>
> >
> > Test thrift
> >
> > start: 1357457502.17
> >
> >   File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> >       File length: 796500
> >
> > end: 1357457509.93
> >
> > elapse: 7.7634768486
> >
> >
> >
> >
> >
> > Test json
> >
> > start: 1357457509.93
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> >       File length: 217252
> >
> > end: 1357457510.01
> >
> > elapse: 0.0743980407715
> >
> >
> As the result above, I can't suffer thrift-serialization is slow like this.
>
> And my question is:
>
>    - Did I have wrong usage with thrift?
>    - Or thrift was not design to transport big data(200k, is it big?)
>
>
> Thanks.
> --
> Vincent.Wen
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message