sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-1771) Investigation FORMAT of the Array/NestedArray/ Set/ Map in Postgres and HIVE.
Date Fri, 21 Nov 2014 16:57:33 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14220290#comment-14220290
] 

Veena Basavaraj edited comment on SQOOP-1771 at 11/21/14 4:57 PM:
------------------------------------------------------------------

postgres - sql:

http://www.postgresql.org/docs/9.1/static/arrays.html
To write an array value as a literal constant, enclose the element values within curly braces
and separate them by commas. (If you know C, this is not unlike the C syntax for initializing
structures.) You can put double quotes around any element value, and must do so if it contains
commas or curly braces. (More details appear below.) Thus, the general format of an array
constant is the following:
{code}
'{ val1 delim val2 delim ... }'
{code}
where delim is the delimiter character for the type, as recorded in its pg_type entry. Among
the standard data types provided in the PostgreSQL distribution, all use a comma (,), except
for type box which uses a semicolon (;). Each val is either a constant of the array element
type, or a subarray. An example of an array constant is:
{code}

'{{1,2,3},{4,5,6},{7,8,9}}'
{code}


Hive representation of data s pluggable with the type ser/deser to use. So there is no one
standard.
Avro is a common standard.

Details of JSON serde is here

http://blog.cloudera.com/blog/2012/12/how-to-use-a-serde-in-apache-hive/
Some references.

http://www.slideshare.net/zshao/hive-serde-and-lazyserde

https://github.com/rcongiu/Hive-JSON-Serde

The default ser/De which is no way easy to understand is here:)

https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java

https://github.com/apache/hive/blob/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java




was (Author: vybs):
Hive representation of data s pluggable with the type ser/deser to use. So there is no one
standard.
Avro is a common standard.

Details of JSON serde is here

http://blog.cloudera.com/blog/2012/12/how-to-use-a-serde-in-apache-hive/
Some references.

http://www.slideshare.net/zshao/hive-serde-and-lazyserde

https://github.com/rcongiu/Hive-JSON-Serde

The default ser/De which is no way easy to understand is here:)

https://github.com/apache/hive/blob/trunk/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinarySerDe.java

https://github.com/apache/hive/blob/trunk/serde/src/test/org/apache/hadoop/hive/serde2/lazybinary/TestLazyBinarySerDe.java



> Investigation FORMAT of the Array/NestedArray/ Set/ Map in Postgres and HIVE.
> -----------------------------------------------------------------------------
>
>                 Key: SQOOP-1771
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1771
>             Project: Sqoop
>          Issue Type: Sub-task
>          Components: sqoop2-framework
>            Reporter: Veena Basavaraj
>             Fix For: 1.99.5
>
>
> update this wiki, which is missing details on the complex types
> https://cwiki.apache.org/confluence/display/SQOOP/Sqoop2+Intermediate+representation#Sqoop2Intermediaterepresentation-Intermediateformatrepresentationproposal



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message