thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David Bennett" <>
Subject RE: How best to add data types esp DateTime and Decimal
Date Thu, 10 Sep 2015 02:30:22 GMT
Not exactly. I should make it clear: there is no Andl API for Thrift to present to users. None
at all. Nada.

Andl is a relational database language. When you compile a program you get a series of relational
variables (SQL: tables) and catalog functions (SQL: stored procedures). Catalog functions
are the only thing exposed by Andl, and my aim is to allow those catalog functions to be exposed
by the Thrift RPC mechanism.

So I really can't 'wrap' anything because the exposed functions are not mine to wrap.

Andl has exactly 5 primitive data types: bool, binary, number, time, text, and 3 structured
types: relation, tuple and user (struct). These are core types and will never need to be versioned.
I might add some (eg integer and double), but these 8 will never change.

Thrift provides an excellent match for 6 of the 8. I need to find a way to support the other
2, number (decimal) and time (datetime), that is as close as possible to being transparent
to the client user. 

I think there are 3 ways to deal with portability, which I call subset, superset and leaky.
Thrift tries to be subset, but i16 and i64 are already leaky. That is, there are languages
which cannot distinguish different sizes of integer and will therefore encounter type errors
in some circumstances if these are used.

The Thrift community has already discussed these issues, and I can see some agonising over
support for 'unsigned'. My view is that Thrift ought to be willing to contemplate leaky support
for a small number of additional primitive types, including decimal and datetime.

For now, my question is: what do I do? How do I use what is already there to provide the best
possible generic support for these types?

[Yes, I found Concourse. It's NoSQL, which is a bit different from what I'm doing. Did you
know the links on this page are broken:]

David M Bennett FACS

Andl - A New Database Language -

-----Original Message-----
From: Jeff Nelson [] 
Sent: Wednesday, 9 September 2015 9:32 PM
Subject: Re: How best to add data types esp DateTime and Decimal

If you want to present your Thrift API directly to users, then you'll probably need to create
some struct that encompasses the information you need for DateTime and Decimal. I'd go with
a struct over a typedef because the former gives you room to evolve in the future if necessary.

With that said, I highly recommend that you DON'T expose your Thrift API directly to users.
Thrift is a GREAT framework for generating rpc code in many languages, but it has to support
the lowest common denominator of languages features so that means it feels clunky and unnatural
in most languages. In the early days, Cassandra exposed their Thrift API directly to users
and it didn't work out well.

Instead, I recommend that you wrap your Thrift API in language specific drivers. This will
allow you to give users libraries that feel natural and adhere to the best practices of the
language they choose. Additionally, this means that you can have logic that automatically
converts whatever is the canonical DateTime class in each language to the thrift struct you're
expecting on the server.

This approach has worked really well for ConcourseDB <>.
For developing the language drivers, thrift makes it so that we don't have to worry about
the RPC stack at all. We simply focus on the business logic of the driver (which is generally
the same across languages...just mapping various types to the appropriate Thrift API calls).
And our users benefit greatly because they don't need to know about Thrift at all (this greatly
reduces the barrier to entry). We're also future proofed in case we ever want to swap out
thrift in the future (though I highly doubt we'd do that because thrift is awesome).


On Tue, Sep 8, 2015 at 10:12 PM, David Bennett <> wrote:

> In order to use Thrift as a remoting API for Andl I have to provide 
> support for two data types: DateTime and Decimal. The question is what 
> strategy to use that will produce the best outcome across common client languages.
> The DateTime data type is yyyy-mm-dd It can use a wire 
> format of i64 (100nsec units since Gregorian 0000, faster) or ISO-8601 (slower).
> The Decimal data type is (at least) 24 decimal digits of precision. It 
> can use a wire format of 2*i64 (special format, faster) or a string (slower).
> The question is not about wire formats but how best to define and 
> implement these types in Thrift for easiest consumption by client 
> software in various languages.
> Is a typedef a useful approach? Or a named struct with a single field?
> How does one make it easy for the client to consume (given that most 
> languages have some kind of preferred type)?
> Regards
> David M Bennett FACS
> Andl - A New Database Language -

View raw message