qpid-proton mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bozo Dragojevic <bo...@digiverse.si>
Subject Re: codec changes
Date Tue, 07 Apr 2015 15:10:37 GMT
On 7. 04. 15 15.38, Alan Conway wrote:
> On Tue, 2015-03-31 at 19:17 +0200, Božo Dragojevič wrote:
>> Given the memory overhead of a pn_data_t before encoding, why not have it
>> own an encode buffer? it could get by with exactly that grow_buffer()
>> callback if ownership is the issue .
> I like this idea a lot. Ownership is an issue for me in Go so I would
> want an alloc/free callback but we could have a non-callback version as
> well (simple wrapper for the callback version with default callbacks)
> for users that don't care about ownership - what do others think?


> Note:
> - alloc called 0 or 1 times, and is passed the exact size needed to encode.

I would hate to have the encoder know a-priori the exact size of allocation.
Especially if we want to, at some point be able to generate a
{LIST|MAP|ARRAY}8 encodings.
I think we can aim for upper bound, but leave out exact-ness.

calculating encoding size for non-composite atoms is easy and cheap so
it could be moved from
encoding to pn_data_put_*() mutators. as long as uuid is part of atom
union, adding another int will not hurt :P
Calculating conservative size for composites is also cheap.

Maybe pn_data_t could even be made to encode as it goes and would thus
not need the extra buffer *and* copying for
variable-sized atoms (strings and binary) and each atom could be reduced
to a offset into the encoded buffer.

> - user is free to allocate more (e.g. doubling strategies)
> - free called after alloc, if and only if alloc is called.
> - separate free instead of single realloc call: encode impl decides
> if/how much data to copy from old to new buffer.

Partial copying of encoded buffers is also problematic, because encoder
has to backtrack to fill out byte sizes for structured types.
So at best we could do a scatter-gather thing, which quickly becomes
another mess as the pn_data_t then needs
a much better arithmetic for offset and size calculations. I really
think we should just offer exactly realloc() for now.


View raw message