thrift-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roger Meier <ro...@bufferoverflow.ch>
Subject Re: Using thrift IDL as the schema for arbitrary JSON messages
Date Wed, 08 Oct 2014 22:49:14 GMT
Hi Peter

Quoting Peter Neumark <neumark.peter@gmail.com>:

> Thanks, Roger!
>
> I'm also posting this to the dev list, because I think that may be more
> appropriate than user@.
Yes, that's why I'm cross posting now.

> The json generator is awesome (any estimate on when it will be merged?),
there are two versions and I like to have one...
I'm ready to commit it as soon as I get proper response on my questions
on that issue. Json Schema is what i would like to have for the future.

> but that's not really what I was working on.
> The goal here is to ready an aribrary JSON message as thrift data.
Yes, I know. However using a schema would simplify this and that's why I
started to talk about the json generator.
-roger

>
> Pete
>
> On Wed, Oct 8, 2014 at 11:47 AM, Roger Meier <roger@bufferoverflow.ch>
> wrote:
>
>> Hi Peter
>>
>> Yes, JSON is worth to add and I see an option by using json schema as
>> mentioned here:
>> https://issues.apache.org/jira/browse/THRIFT-2360
>>
>> best!
>> -roger
>>
>>
>> Quoting Peter Neumark <neumark.peter@gmail.com>:
>>
>>  Hi all,
>>>
>>> At Prezi, we currently use Thrift for a lot of the internal communication
>>> between services. A lot of engineers would like to keep using HTTP + JSON
>>> as means of communicating between services, but with some sort of schema.
>>>
>>> After evaluating several alternatives, the best idea we had so far was to
>>> use the thrift IDL as the schema, and beef up the existing
>>> TSimpleJSONProtocol code to read arbitrary JSON messages.
>>>
>>> What I'm most interested in is whether or not someone else is already
>>> doing
>>> this?
>>>
>>> I have a very early stage python implementation of such a protocol here:
>>> https://github.com/neumark/thriftmodel/blob/master/thriftmodel/
>>> TFlexibleJSONProtocol.py
>>>
>>> We'll probably create an implementation for Java as well.
>>> The most difficult issue with reading JSON is that a lot of existing JSON
>>> protocols allow for several different types in a field. My solution was to
>>> have an "unboxed union". To illustrate this with an example, if we accept
>>> the following messages:
>>>
>>> {"id": 1, "expires": 32324234234}
>>> {"id": 1, "expires": "2014.10.07 10:34"}
>>> {"id": 1, "expires": false}
>>>
>>> Then our Thrift definition would be
>>>
>>> // Note: this is an unboxed union
>>> union { // or struct in my implementation because python has no union yet
>>>   1: i64 unix_timestamp,
>>>   2: string date_string,
>>>   3: bool expiration_switch
>>> }
>>>
>>> struct msg {
>>>   1: i64 id,
>>>   2: expiry expires
>>> }
>>>
>>> The "expires" field must be read "unboxed" so to speak.
>>> What's the best option for encoding this in the thrift IDL?
>>> Has anyone considered adding annotations to the IDL for similar purposes?
>>>
>>> Thanks,
>>> Peter
>>>
>>
>>
>>



Mime
View raw message