gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lewis John McGibbney (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (GORA-206) Verify storage and retrieval of Avro null-single-type Union data type within Gora-Cassandra
Date Mon, 04 Mar 2013 01:24:12 GMT

    [ https://issues.apache.org/jira/browse/GORA-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591901#comment-13591901
] 

Lewis John McGibbney edited comment on GORA-206 at 3/4/13 1:23 AM:
-------------------------------------------------------------------

Looking at this, this morning I see that we've introduced a number of bugs such as

* fields for embedded records are now not handled correctly in new GoraCompiler e.g
{code}
{"name": "protocolStatus", "type": ["null", {
            "name": "ProtocolStatus",
            "type": "record",
            "namespace": "org.apache.nutch.storage",
            "fields": [
                {"name": "code", "type": "int"},
                {"name": "args", "type": {"type": "array", "items": "string"}},
                {"name": "lastModified", "type": "long"}
            ]
            }]}
{code}
is simply compiled down into
{code}
{\"name\":\"protocolStatus\",\"type\":[\"null\",\"protocolstatus\"]},
{code} 

* additionally fields containing array and map values are now not handled correctly in GoraCompiler
e.g.

{code}
{"name": "headers", "type": {"type": "map", "values": "string"}}
{code}

is incorrectly compiled down into

{code}
{\"name\":\"metadata\",\"type\":\"map\"}
{code}

We need to sort these two cases at a minimum. These are blockers.

I'll work on these two issues and hopefully attach a working v3 patch.
                
      was (Author: lewismc):
    Looking at this, this morning I see that we've introduced a number of bugs such as

* fields for embedded records are now not handled correctly in new GoraCompiler e.g
{code}
{"name": "protocolStatus", "type": ["null", {
            "name": "ProtocolStatus",
            "type": "record",
            "namespace": "org.apache.nutch.storage",
            "fields": [
                {"name": "code", "type": "int"},
                {"name": "args", "type": {"type": "array", "items": "string"}},
                {"name": "lastModified", "type": "long"}
            ]
            }]}
{code}
is simply compiled down into
{code}
{\"name\":\"protocolStatus\",\"type\":[\"null\",\"protocolstatus\"]},
{code} 

* additionally fields containing array and map values are now not handled correctly in GoraCompiler
e.g.
{code}
{"name": "headers", "type": {"type": "map", "values": "string"}}
{code}
is incorrectly compiled down into
{code}
{\"name\":\"metadata\",\"type\":\"map\"}
{code}

We need to sort these two cases at a minimum. These are blockers.

I'll work on these two issues and hopefully attach a working v3 patch.
                  
> Verify storage and retrieval of Avro null-single-type Union data type within Gora-Cassandra
> -------------------------------------------------------------------------------------------
>
>                 Key: GORA-206
>                 URL: https://issues.apache.org/jira/browse/GORA-206
>             Project: Apache Gora
>          Issue Type: Sub-task
>          Components: storage-cassandra
>    Affects Versions: 0.3
>            Reporter: Renato Javier Marroquín Mogrovejo
>            Assignee: Renato Javier Marroquín Mogrovejo
>              Labels: gora-cassandra, gora-core
>             Fix For: 0.3
>
>         Attachments: GORA-206.v1.patch, GORA-206.v2.patch, GORA-206.v3.patch
>
>
> The necessary features should be added to confirm that we are able to support Avro Union
data types.
> This referes specifically to null-single-type unions. We will open another issue to address
the multi-type unions.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message