thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject Re: heterogeneous collections
Date Mon, 03 May 2010 05:28:48 GMT

Some thoughts on Thrift & its encoding style:
- Thrift, at the wire level, encodes values with an accompanying type.
- The thrift RPC demarshallers are told the type they are expecting
- The received message may not be the expected type.
- What happens if there is an incompatibility? Thrift appears to only
define two situations:
  1. a structure expects less fields than are actually sent; expected
behavior is to discard the fields
  2. a structure expects more fields than are actually sent; expected
behavior is to leave the unsent fields uninitialized (or set to some
default value).
- Cases which might have been defined but are not are things such as:
expect a i32 but was on the wire as i16.

In some sense, at the wire level, Thrift can be thought of as sending
messages that are
* scalars (bool,byte,i16,i32,i64,double,strings?)
* structs that are maps from field numbers to values
* lists (&sets) that are sequences of values
* maps that are sequences of tuples.

On the wire, Thrift is permissive enough to send a value of the type
list { struct {1: I16, 2:I32}, struct{ 1:double, 5:string}}
In that sense, it will support list<any>.

However, in a typed language (and probably in any language other than one
with duck-typing), it will be difficult (or impossible) to convert the
received object into an acceptable object.

[[ This also suggests that there is a possible compression available here:
specify complete types as arguments to containers, as opposed to repeating
them for each element. Taken to its conclusion, however, this requires
sending the type signature in advance of the actual message, as opposed to
interleaved with it.

Note that, with that one exception, it seems unclear where you could save
logical space in the type-encoding overhead, and still retain the ability
to change the number of fields & their types and then detect that there is
an incompatiblity.

You may, of course, be able to compress the logical type information, as
well as the values transmitted.]]

So, the problem with list<any> is not necessarily the wire encoding, but
specifying the expected type -- what should the demarshaller turn it into?

> Hi,
> I'd like to use an heterogeneous collection, e.g. list<any>, in a service.
> I see there's been some work/discussion on this earlier,
> Now THRIFT-110 is committed and claims to incorporate THRIFT-122.
> I've looked around but couldn't find the answer... how do I declare an
> heterogeneous collection in a .thrift definition?
> alex

View raw message