thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Meyer <joel.me...@gmail.com>
Subject Re: Thrift containers for java
Date Fri, 27 Aug 2010 03:57:22 GMT
On Thu, Aug 26, 2010 at 8:39 PM, Sumanta Bhowmik <sumanta.bhowmik@guavus.com
> wrote:

> Hi
>
> I am using thrift for passing data among multiple layers. Most of our data
> is streaming, it is array of primitives like doubles, ints. Thrift requires
>  me to use a List in the IDL, which on the Java side translates to ArrayList
> of Integers. I find two problems with this approach
>
> 1. Integers are heavier than ints are, so it bloats up the memory.
> Something like an int[] rather than ArrayList<Integer> would have been
> perfect for our kind of usecase.
> 2. A list of integers is read integer by integer, rather than all of it in
> a single shot. This bring down the performance as the 2million integers mean
> that many method calls.
>
> Is there a clean way of passing primitive arrays? Thrift provide byte which
> is basically byte[], so I guess there should be some way for int[] or
> double[] as well.
>

Not currently. You could wrap a byte[] with a ByteBuffer, call
ByteBuffer.asIntBuffer and use the returned IntBuffer 'view' to write
primitive ints. As long as you're careful about allocating the right sized
byte[] and reading it the same way you write it (with an IntBuffer view of a
ByteBuffer) you could use the byte[] type for transporting and accessing
your ints while avoiding the overhead of auto-boxing. You'll still have 2M
calls to ByteBuffer.readInt, so you may not save much overhead in the way of
method calls.

Cheers,
Joel


>
> Regards
> Sumanta
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message