thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghava Mutharaju <m.vijayaragh...@gmail.com>
Subject Re: C++ deserialization - record length?
Date Fri, 23 Jul 2010 21:22:11 GMT
Hi Bryan,

Thank you for the reply. In that case, I will try the first option of
preceding the data with its length. How can this hookup (2nd solution) be
done?

In the first solution, there would be issues regarding the no of bytes to be
used to save the size of the data isn't it. If I use a byte, then I can only
keep track of data (object) of max size 2^8-1. Isn't this a problem?

Thank you.

Regards,
Raghava.

On Fri, Jul 23, 2010 at 5:07 PM, Bryan Duxbury <bryan@rapleaf.com> wrote:

> 1) If and only if they had the exact same contents. Which means, probably
> not.
>
> The usual solution to this problem is to serialize into a buffer and write
> the size to the file before writing the serialized data. An alternative is
> to hook your deserializer up directly to the file stream and letting Thrift
> figure out where stuff starts and ends, though this can be a bit tricky.
>
> On Fri, Jul 23, 2010 at 1:47 PM, Raghava Mutharaju <
> m.vijayaraghava@gmail.com> wrote:
>
> > Hi Bryan,
> >
> > Thank you for the reply. In that case, I have couple of more questions.
> >
> > 1) Would the buffers of 2 Employee objects have same size?
> > 2) If the answer to above question is 'yes', then I need to somehow pass
> > the
> > buffer size to the other application which would deserialize these list
> of
> > objects. Would writing this value to another file be a good option?
> >
> > Thank you.
> >
> > Regards,
> > Raghava.
> >
> > On Fri, Jul 23, 2010 at 4:38 PM, Bryan Duxbury <bryan@rapleaf.com>
> wrote:
> >
> > > Serialized thrift objects aren't fixed size, nor do their in-memory
> > > representations reflect their serialized representation. Unless there's
> > > more
> > > magic to sizeof() than I was expecting, you need to look at the size of
> > the
> > > buffer after you write out, not the size of the struct.
> > >
> > > On Fri, Jul 23, 2010 at 1:27 PM, Raghava Mutharaju <
> > > m.vijayaraghava@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I have serialized couple of Employee objects (binary protocol) and
> > saved
> > > > the
> > > > byte array into a file. I am using C++ to deserialize. I would reach
> > > bytes
> > > > of one Employee from the file each time and give it to read() method.
> > If
> > > I
> > > > use sizeof(Employee), it does not give the right size.
> > > >
> > > > Some statistics from a sample run:
> > > >
> > > > Total Employee objects serialized = 10
> > > > Total bytes in the file = 690. So each Employee object size should be
> > 69
> > > > bytes.
> > > > sizeof(Employee) gives 96.
> > > >
> > > > This seems to be a frequently used operation. How can this be done?
> > > > I would read each object, fill the TMemoryBuffer and use it to
> > construct
> > > > TBinaryProtocol.
> > > >
> > > > Thank you.
> > > >
> > > > Regards,
> > > > Raghava.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message