calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <>
Subject Re: [Avatica] Separating wire message from service implementation
Date Fri, 07 Aug 2015 16:23:40 GMT
:) I hear you. I don't think I have anything putting me in one camp or 
the other aside from potential worries.

I can respect the Thrift "doing too much" notion -- kind of half of my 
concern in using something like Thrift just to get object serialization 
stuff (now I'm thinking I should help separate that out within thrift!) 
I've used Thrift since the 0.6 timeframe and I don't remember once 
coming across an issue where object serialization wasn't handled correctly.

I'll have to poke around pb3 docs some more to see if I can find some 
talk on compatibility. I think at this point I'm leaning towards 
protobuf, but we'll see.

Thanks again for taking the time to write up your thoughts, Jesse!

Jesse Yates wrote:
> Yup, totally get it. Honestly, I have no idea what the compat story is
> supposed to be for proto3. At the same time, I've got no hands on
> experience w/ Thrift either, beyond dabbling a little bit years ago, so I
> don't know what their on-the-wire-compat story is either :) I'm not as huge
> of a fan of thrift in general though because it tries to do too much IMO
> If we do go the thrift route, but want to support grpc, we could do the
> work to make a thrift adapter, so picking one or the other now doesn't
> really preclude doing cool stuff later.
> In short, it doesn't really matter? At least not to me.
> On Thu, Aug 6, 2015 at 3:57 PM Josh Elser<>  wrote:
>> Thanks for the feedback, Jesse!
>> To clarify, the compatibility I'm most worried about is backwards compat
>> in future versions of protobuf3. I hadn't considered v2 support into the
>> equation -- figured v3 is the starting point like you say. I know the
>> protobuf 2.4 to 2.5 upgrade in Hadoop/HBase land was rather scary that
>> required a lot of lock-step movement across many big projects. This is
>> my biggest fear with protobuf.
>> Despite fear of derailing conversation, grpc definitely has some nice
>> features built in (and implementing a service is definitely slick). I
>> think it's definitely a "later" since the single endpoint presently used
>> doesn't really require a complex service definition (with evolving args,
>> etc). My gut reaction is to wait to adopt such software until there is a
>> definite understanding of what the service would look like (which I
>> assume will become clearer as development continues).
>> Jesse Yates wrote:
>>> +1 for proto3. mostly for the later move to using grpc, which lets us
>>> leverage a battle tested framework for doing real protobuf services with
>>> fast streaming (how nice!).
>>> proto3 can be backwards compat in some ways (eg. map implementation done
>>> manually in proto2). I don't think its a huge issue since there are
>>> currently no backwards compat guarantees anyways AFAIK.
>>> On Thu, Aug 6, 2015 at 3:11 PM Josh Elser<>   wrote:
>>>> Hi,
>>>> I have some ideas which I'd like to try to working on to improve
>>>> Avatica. My direct interest is in support of the Phoenix Query Server
>>>> which builds on top of Avatica.
>>>> One of the big areas of improvement that stands out to me is the
>>>> Jackson-annotated Response/Request classes. First, we'd want to make
>>>> sure these classes are stable (in terms of fields/attributes) so that,
>>>> as each object evolves, we prevent existing clients from breaking.
>>>> Second, it's desirable that we have some representation of these objects
>>>> in a manner that's accessible to non-Java clients.
>>>> Ideally, preserving JSON as the serialized representation of these
>>>> classes is also desirable as it's always pleasant to actually be able to
>>>> read the data going over the wire when necessary.
>>>> Given these goals, two tools stood out to me as candidates that would
>>>> solve this problem well without extra effort in the transport: Thrift
>>>> and Protobuf v3.
>>>> Using Thrift only for object generation is a bit overkill, but this is
>>>> an area of Thrift that has been rock-solid in the years that I've used
>>>> it. Classpath issues with Thrift are always a concern as multiple
>>>> versions of Thrift tend to break terribly (at least when performing
>>>> Thrift RPCs -- I wouldn't necessarily trust it to be safe for only
>>>> object serialization). It's easy to serialize Thrift objects to JSON and
>>>> we can generate these classes for other languages.
>>>> I would call Protobuf v3 very young, although it does appear to fit the
>>>> bill. Protobuf v2 has had a track record of being a pain when different
>>>> versions of the library exist. For example, the Java classes generated
>>>> by protobuf 2.x would fail to work with protobuf 2.y. I assume there's
>>>> not enough evidence on protobuf v3 to say whether this will continue to
>>>> be a problem. JSON-representation of objects and cross-language support
>>>> are both natively supported (best as I can tell). Hypothetically,
>>>> adopting protobufv3 could also have some extra benefits down the road
>>>> such as leveraging grpc to replace Jetty, but that is an entirely
>>>> different discussion for later :)
>>>> If you've gotten this far, I'd humbly ask for any opinions on the
>>>> technology choices. Any opinions for or against one or the other? Any
>>>> similar experiences?
>>>> Thanks!
>>>> - Josh

View raw message