calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <>
Subject Re: [Avatica] Separating wire message from service implementation
Date Thu, 06 Aug 2015 22:57:04 GMT
Thanks for the feedback, Jesse!

To clarify, the compatibility I'm most worried about is backwards compat 
in future versions of protobuf3. I hadn't considered v2 support into the 
equation -- figured v3 is the starting point like you say. I know the 
protobuf 2.4 to 2.5 upgrade in Hadoop/HBase land was rather scary that 
required a lot of lock-step movement across many big projects. This is 
my biggest fear with protobuf.

Despite fear of derailing conversation, grpc definitely has some nice 
features built in (and implementing a service is definitely slick). I 
think it's definitely a "later" since the single endpoint presently used 
doesn't really require a complex service definition (with evolving args, 
etc). My gut reaction is to wait to adopt such software until there is a 
definite understanding of what the service would look like (which I 
assume will become clearer as development continues).

Jesse Yates wrote:
> +1 for proto3. mostly for the later move to using grpc, which lets us
> leverage a battle tested framework for doing real protobuf services with
> fast streaming (how nice!).
> proto3 can be backwards compat in some ways (eg. map implementation done
> manually in proto2). I don't think its a huge issue since there are
> currently no backwards compat guarantees anyways AFAIK.
> On Thu, Aug 6, 2015 at 3:11 PM Josh Elser<>  wrote:
>> Hi,
>> I have some ideas which I'd like to try to working on to improve
>> Avatica. My direct interest is in support of the Phoenix Query Server
>> which builds on top of Avatica.
>> One of the big areas of improvement that stands out to me is the
>> Jackson-annotated Response/Request classes. First, we'd want to make
>> sure these classes are stable (in terms of fields/attributes) so that,
>> as each object evolves, we prevent existing clients from breaking.
>> Second, it's desirable that we have some representation of these objects
>> in a manner that's accessible to non-Java clients.
>> Ideally, preserving JSON as the serialized representation of these
>> classes is also desirable as it's always pleasant to actually be able to
>> read the data going over the wire when necessary.
>> Given these goals, two tools stood out to me as candidates that would
>> solve this problem well without extra effort in the transport: Thrift
>> and Protobuf v3.
>> Using Thrift only for object generation is a bit overkill, but this is
>> an area of Thrift that has been rock-solid in the years that I've used
>> it. Classpath issues with Thrift are always a concern as multiple
>> versions of Thrift tend to break terribly (at least when performing
>> Thrift RPCs -- I wouldn't necessarily trust it to be safe for only
>> object serialization). It's easy to serialize Thrift objects to JSON and
>> we can generate these classes for other languages.
>> I would call Protobuf v3 very young, although it does appear to fit the
>> bill. Protobuf v2 has had a track record of being a pain when different
>> versions of the library exist. For example, the Java classes generated
>> by protobuf 2.x would fail to work with protobuf 2.y. I assume there's
>> not enough evidence on protobuf v3 to say whether this will continue to
>> be a problem. JSON-representation of objects and cross-language support
>> are both natively supported (best as I can tell). Hypothetically,
>> adopting protobufv3 could also have some extra benefits down the road
>> such as leveraging grpc to replace Jetty, but that is an entirely
>> different discussion for later :)
>> If you've gotten this far, I'd humbly ask for any opinions on the
>> technology choices. Any opinions for or against one or the other? Any
>> similar experiences?
>> Thanks!
>> - Josh

View raw message