sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-489) Cannot define partition keys for Hive tables created through Sqoop
Date Fri, 25 May 2012 06:10:22 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283148#comment-13283148
] 

Jarek Jarcec Cecho commented on SQOOP-489:
------------------------------------------

I believe that adjusting generated "Create table" statement is only half of the job. We also
need to stop importing those columns, because otherwise we would end up with different number
of columns in imported file and in hive which might lead to a huge confusion on user side.

Jarcec
                
> Cannot define partition keys for Hive tables created through Sqoop
> ------------------------------------------------------------------
>
>                 Key: SQOOP-489
>                 URL: https://issues.apache.org/jira/browse/SQOOP-489
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.1-incubating
>            Reporter: Kathleen Ting
>
> By enabling the table option, Sqoop includes every column in the table in the create
table query, and by enabling the hive-partition-key option, Sqoop blindly appends the "partitioned
by" clause. Now if you specify one of columns in the table in the hive-partition-key, this
will cause a syntax error in Hive.
> For example, if we have a table 'FOO' that has columns 'I' and 'J':
> sqoop create-hive-table --table FOO ...
> will generate the following Hive query:
> CREATE TABLE IF NOT EXISTS `FOO` ( `I` STRING, `J` STRING)
> Now if we add "--hive-partition-key I" to the command, Sqoop generates the following
query:
> CREATE TABLE IF NOT EXISTS `FOO` ( `I` STRING, `J` STRING) PARTITIONED BY (I STRING)
> The problem is that since 'I' is defined twice (once in CRATE TABLE and once in PARTITIONED
BY), this is a syntax error in Hive.
> This correct query would be something like:
> CREATE TABLE IF NOT EXISTS `FOO` (`J` STRING) PARTITIONED BY (I STRING)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message