sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1395) Potential naming conflict in Avro schema
Date Thu, 04 Sep 2014 09:26:52 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14121166#comment-14121166
] 

ASF subversion and git services commented on SQOOP-1395:
--------------------------------------------------------

Commit 500b75ae4b990fd0e28ea4f3cb5a93d1e68b0d89 in sqoop's branch refs/heads/trunk from [~jarcec]
[ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=500b75a ]

SQOOP-1395: Potential naming conflict in Avro schema

(Qian Xu via Jarek Jarcec Cecho)


> Potential naming conflict in Avro schema
> ----------------------------------------
>
>                 Key: SQOOP-1395
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1395
>             Project: Sqoop
>          Issue Type: Sub-task
>          Components: tools
>            Reporter: Qian Xu
>            Assignee: Qian Xu
>            Priority: Minor
>         Attachments: SQOOP-1395.patch
>
>
> If you import a table "users". Sqoop will generate an entity class named "users.java".
The class will be compiled, submitted and used by a mapreduce job. If the target file format
is Avro or Parquet, an Avro schema will be generated as well. According to Avro specification,
the entity class is described as "record", the name of the "record" is "users".
> For Parquet file format handling, we use the Kite SDK to manage Parquet file reading
and writing with minimal efforts. Kite requires an Avro schema and all data records to be
packed into GenericRecord instances. There will be a problem here. Kite will read the schema
first and try to instantiate a record regarding its name. In this case, Kite will try to instantiate
a "users" class. Unfortunately, there is a "users.java" out there. This will cause mapreduce
job fail. 
> The patch proposes to change the {{AvroSchemaGenerator}} class. Record name will have
a prefix. In this example, the record name of "users.java" will be changed to "sqoop_import_users".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message