drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Timothy Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-19) Build a JSON scanner that does schema discovery
Date Sun, 13 Jan 2013 17:12:12 GMT

    [ https://issues.apache.org/jira/browse/DRILL-19?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13552242#comment-13552242
] 

Timothy Chen commented on DRILL-19:
-----------------------------------

I've put latest code on github that supports proto output.

I've only tested several simple situations and I can definitely spend more time testing much
complex json schemas.

Currently when outputing proto schema with mixing lists and object, I will assign the list
schema to a fixed message type ListSchema that has just a byte field that stores the embedded
schema.

I haven't write the code yet to write embedded schema and deserialize the embedded schema,
since it's just the proto definition not the actual message contents.

Do you want me to continue to turn the data records into proto messages or you have something
other in mind?

                
> Build a JSON scanner that does schema discovery
> -----------------------------------------------
>
>                 Key: DRILL-19
>                 URL: https://issues.apache.org/jira/browse/DRILL-19
>             Project: Apache Drill
>          Issue Type: New Feature
>            Reporter: Jacques Nadeau
>            Assignee: Timothy Chen
>
> Build a JSON scanner that reads a file and converts it into two parts: a stream of records
and a schema which reflects the schema of the records.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message