thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Suggestion/ Complaint from long time thrift user
Date Sun, 10 Aug 2014 18:12:28 GMT
Hello all,

Let me get this out of the way. I love thrift. I have used it for a long
time for internal projects, as well as open source projects like Apache
Cassanra and Hive. Using thrift as a user is great, the thrift objects and
themselves are largely wire compatible so that I can typically have a
thrift 0.6.X service getting a message from a 0.7.X client and it usually
just works.

Thrift has one hugely annoying problem to me on the java side of things.
Even though the wire format is largely forward and reverse compatible
across a number of versions the API's change very frequently and this
causes a number of major frustrations for me.

This pretty much sums it all up:
https://twitter.com/edwardcapriolo/status/497835070846435328

This tweet was a result of an entire unproductive 2 days of head bashing
trying to get two projects that both use thrift to work together. Here is
the skinny...

Apache Hive has a very nice JDBC transport built on top of thrift.  Thrift
0.9.X
Asytanax is a Client for Apache build against Cassandra 1.2 that was built
against thrift 0.7.X

In many cases in the Java world, dynamic binding smooths over versions, but
with thrift it almost never seems to work out that way. When I want to use
Hive and Astyanax in the same project it blows up at runtime. This puts
users who are not full time open source developers in a huge bind. You have
to stop whatever you are doing and attempt to find a way to get your thrift
versions equal or close enough so your projects are happy.

In this case I found that newer versions of Cassandra are compiled against
thrift 0.9.1 however there was no version of Cassandra build against 0.9.0.
So between the two projects Hive and Cassandra I am not easily able to find
a hybrid of versions that will work together. Also I want to run specific
versions not just ones I am forced to run for classpath compatibility.

So, I figured let me role the dice, maybe cassandra 2.0 with thrift 0.9.1
and hive 12.0-cloudera-hack with version thrift 0.9.0 could live happily
together. I patched up astyanax (https://github.com/edwardcapriolo/astyanax)
to make this happen.

Fail:
https://twitter.com/edwardcapriolo/status/497835070846435328

I totally understand that thrift is not a magic bullet, when protobuf
changed from 2.4 to 2.5 they had the exact same issue (wire compatibility
but not API compatible), but IMHO the API of protobuf is more stable.  I
only remember 1 of these in 4-5 years with protobuf whereas I run into this
brick wall constantly with thrift.

My far flung suggestion: The API for thrift need to be versioned e.g.
org.apache.thrift.thrift10.Server and org.apache.thrift.thrift11.server so
that when a user finds two projects not at the exact same thrift revision
they can work together in the same JVM runtime without doing
classloaders/jar jar/ osgi etc. Because at this point I find it very
difficult to use thrift without a company wide mandate of "Everyone user
version x" and even with a company wide mandate if I need to integrated
with any software not made at my company I inevitably run into the same
problem.

Thanks for listening.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message