calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesse Yates <>
Subject Re: [Avatica] Separating wire message from service implementation
Date Fri, 07 Aug 2015 15:58:43 GMT
Yup, totally get it. Honestly, I have no idea what the compat story is
supposed to be for proto3. At the same time, I've got no hands on
experience w/ Thrift either, beyond dabbling a little bit years ago, so I
don't know what their on-the-wire-compat story is either :) I'm not as huge
of a fan of thrift in general though because it tries to do too much IMO

If we do go the thrift route, but want to support grpc, we could do the
work to make a thrift adapter, so picking one or the other now doesn't
really preclude doing cool stuff later.

In short, it doesn't really matter? At least not to me.

On Thu, Aug 6, 2015 at 3:57 PM Josh Elser <> wrote:

> Thanks for the feedback, Jesse!
> To clarify, the compatibility I'm most worried about is backwards compat
> in future versions of protobuf3. I hadn't considered v2 support into the
> equation -- figured v3 is the starting point like you say. I know the
> protobuf 2.4 to 2.5 upgrade in Hadoop/HBase land was rather scary that
> required a lot of lock-step movement across many big projects. This is
> my biggest fear with protobuf.
> Despite fear of derailing conversation, grpc definitely has some nice
> features built in (and implementing a service is definitely slick). I
> think it's definitely a "later" since the single endpoint presently used
> doesn't really require a complex service definition (with evolving args,
> etc). My gut reaction is to wait to adopt such software until there is a
> definite understanding of what the service would look like (which I
> assume will become clearer as development continues).
> Jesse Yates wrote:
> > +1 for proto3. mostly for the later move to using grpc, which lets us
> > leverage a battle tested framework for doing real protobuf services with
> > fast streaming (how nice!).
> >
> > proto3 can be backwards compat in some ways (eg. map implementation done
> > manually in proto2). I don't think its a huge issue since there are
> > currently no backwards compat guarantees anyways AFAIK.
> >
> >
> > On Thu, Aug 6, 2015 at 3:11 PM Josh Elser<>  wrote:
> >
> >> Hi,
> >>
> >> I have some ideas which I'd like to try to working on to improve
> >> Avatica. My direct interest is in support of the Phoenix Query Server
> >> which builds on top of Avatica.
> >>
> >> One of the big areas of improvement that stands out to me is the
> >> Jackson-annotated Response/Request classes. First, we'd want to make
> >> sure these classes are stable (in terms of fields/attributes) so that,
> >> as each object evolves, we prevent existing clients from breaking.
> >> Second, it's desirable that we have some representation of these objects
> >> in a manner that's accessible to non-Java clients.
> >>
> >> Ideally, preserving JSON as the serialized representation of these
> >> classes is also desirable as it's always pleasant to actually be able to
> >> read the data going over the wire when necessary.
> >>
> >> Given these goals, two tools stood out to me as candidates that would
> >> solve this problem well without extra effort in the transport: Thrift
> >> and Protobuf v3.
> >>
> >> Using Thrift only for object generation is a bit overkill, but this is
> >> an area of Thrift that has been rock-solid in the years that I've used
> >> it. Classpath issues with Thrift are always a concern as multiple
> >> versions of Thrift tend to break terribly (at least when performing
> >> Thrift RPCs -- I wouldn't necessarily trust it to be safe for only
> >> object serialization). It's easy to serialize Thrift objects to JSON and
> >> we can generate these classes for other languages.
> >>
> >> I would call Protobuf v3 very young, although it does appear to fit the
> >> bill. Protobuf v2 has had a track record of being a pain when different
> >> versions of the library exist. For example, the Java classes generated
> >> by protobuf 2.x would fail to work with protobuf 2.y. I assume there's
> >> not enough evidence on protobuf v3 to say whether this will continue to
> >> be a problem. JSON-representation of objects and cross-language support
> >> are both natively supported (best as I can tell). Hypothetically,
> >> adopting protobufv3 could also have some extra benefits down the road
> >> such as leveraging grpc to replace Jetty, but that is an entirely
> >> different discussion for later :)
> >>
> >> If you've gotten this far, I'd humbly ask for any opinions on the
> >> technology choices. Any opinions for or against one or the other? Any
> >> similar experiences?
> >>
> >> Thanks!
> >>
> >> - Josh
> >>
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message