calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Should Jackson-annotated POJOs stick around
Date Tue, 11 Aug 2015 18:47:34 GMT
I'll admit that I don't understand why protocol buffers is being equated 
with the transport only.

The biggest reason I think Avatica can benefit from using software like 
protobuf is that it makes handling a shift in the API substantially 
easier. For example, what happens when a new field is added to a 
Request? What if you receive a Request that doesn't have a field that 
you expected it to (old client)? This is the problem that I'm trying to 
solve. Regardless of whether this is coming in over the Java API or an 
HTTP connection, the version of the Request or Response (and the actual 
attributes that it contains) are near guaranteed to change.

I don't really care what the bytes look like going over the wire. That's 
just a side-effect to which my only concern is to meet any desires for 
readability that may exist.

Does that make sense?

Julian Hyde wrote:
> The conventional way of implementing a driver uses an end-to-end
> design: "just use the MySQL driver and have a MySQL server on the
> other end and it will just work".
>
> With Avatica I'm trying to introduce a layered approach (inspired by
> the OSI model) where we specify what happens at several levels. We
> specify a wire format, and we specify the Java API.
>
> An application doesn't have to to use all of the levels. Some
> applications might want to use the Java API but plug in a different
> transport. Other applications might want to build a C or Python API on
> top of the wire format.
>
> My goal is to ensure that the driver stack can be re-used for
> different database engines, and from different languages, and that a
> driver in local mode is substantially the same as a driver in remote
> mode.
>
> Notice I said "a wire format". I don't mind if there are other wire
> formats. But I would be concerned if one transport muscled in and left
> its mark up and down the whole stack.
>
>
> On Tue, Aug 11, 2015 at 10:58 AM, Josh Elser<josh.elser@gmail.com>  wrote:
>> Ok, that's where the confusion is coming in :). I envision protocol buffer
>> objects replacing the POJOs in both the wire api and the Meta API.
>>
>> Trying to do both seems without gain for me (again, aside from preserving
>> backwards compatibility with existing releases). If that requires a
>> different Meta interface+impl (or a translation layer), that's fine too --
>> just extra work (and a slight performance hit on whichever is treated as the
>> reference object).
>>
>> Does that make sense?
>>
>> Julian Hyde wrote:
>>> Yes, the POJOs are needed. The service layer (which for a particular
>>> client may or may not be backed by RPC) consists of methods that take
>>> complex arguments and return complex results. Those arguments and
>>> results are expressed as POJOs.
>>>
>>> One example:
>>>
>>> Frame fetch(StatementHandle h, List<TypedValue>   parameterValues, long
>>> offset,
>>>       int fetchMaxRowCount);
>>>
>>> class Frame {
>>>     public final long offset;
>>>     public final boolean done;
>>>     public final Iterable<Object>   rows;
>>> }
>>>
>>> public class TypedValue {
>>>     public final ColumnMetaData.Rep type;
>>>     public final Object value;
>>> }
>>>
>>> The POJOs are definitely needed for the Java API Meta and appear in
>>> some form by whatever the wire protocol is.
>>>
>>> Julian
>>>
>>>
>>>
>>>
>>> On Tue, Aug 11, 2015 at 10:16 AM, Josh Elser<josh.elser@gmail.com>   wrote:
>>>> Pulling this out to email to avoid cluttering JIRA. I feel like I might
>>>> not
>>>> be on the same page.
>>>>
>>>> I see CALCITE-839 and CALCITE-840 being one in the same change, or at
>>>> least
>>>> the root cause being solved by it.
>>>>
>>>> Julian, I'm getting the impression that you envision protocol buffer
>>>> encoding being just another option for encoding requests and responses.
>>>> My
>>>> opinion is that using protocol buffers to define these requests and
>>>> responses completely invalidates the need to support these POJOs. These
>>>> objects should be usable cross-language, so aside from support the
>>>> releases
>>>> of Calcite which shipped only these POJOs, I don't see a need to maintain
>>>> them.
>>>>
>>>> I am admittedly hedging my bet that the PB devs will release a new
>>>> version
>>>> that has the advertised JSON-esque serialization format (instead of just
>>>> a
>>>> binary format). If this ultimately falls through, POJOs that just wrap
>>>> the
>>>> PB classes could also be done.
>>>>
>>>> I just wanted to make sure I'm not dancing by myself and that we're all
>>>> in
>>>> agreement on a general direction.
>>>>
>>>> -------- Original Message --------
>>>> Subject: [jira] [Commented] (CALCITE-839) Remove Jackson annotations from
>>>> POJO classes in Meta
>>>> Date: Tue, 11 Aug 2015 05:41:45 +0000 (UTC)
>>>> From: Julian Hyde (JIRA)<jira@apache.org>
>>>> Reply-To: dev@calcite.incubator.apache.org
>>>> To: issues@calcite.incubator.apache.org
>>>>
>>>>
>>>>       [
>>>>
>>>> https://issues.apache.org/jira/browse/CALCITE-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681279#comment-14681279
>>>> ]
>>>>
>>>> Julian Hyde commented on CALCITE-839:
>>>> -------------------------------------
>>>>
>>>> Well then, I've assigned CALCITE-840 to you.
>>>>
>>>>> Remove Jackson annotations from POJO classes in Meta
>>>>> ----------------------------------------------------
>>>>>
>>>>>                   Key: CALCITE-839
>>>>>                   URL: https://issues.apache.org/jira/browse/CALCITE-839
>>>>>               Project: Calcite
>>>>>            Issue Type: Bug
>>>>>            Components: avatica
>>>>>              Reporter: Julian Hyde
>>>>>              Assignee: Julian Hyde
>>>>>
>>>>> The Meta interface contains several POJO classes that represent RPC
>>>>> requests or responses. Currently a few of those classes have Jackson
>>>>> annotations such as @JsonCreator, @JsonProperty to help Jackson
>>>>> serialize
>>>>> the POJO to JSON and de-serialize from JSON to the object.
>>>>> As [~ndimiduk] pointed out in
>>>>>
>>>>> http://mail-archives.apache.org/mod_mbox/incubator-calcite-dev/201503.mbox/%3CCANZa=GvKGD+BKJ4+eJMuO6iVhS+OkGSkG1VwDAZCy-ZijyyyPw@mail.gmail.com%3E
>>>>> these annotations are a "code smell" and should be removed. It makes
it
>>>>> look
>>>>> as if Jackson is the only possible transport, which is not the case.
We
>>>>> can
>>>>> continue to use Jackson as a transport, just specify the mappings
>>>>> elsewhere,
>>>>> not as annotations.
>>>>
>>>>
>>>>
>>>> --
>>>> This message was sent by Atlassian JIRA
>>>> (v6.3.4#6332)

Mime
View raw message