sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jarek Jarcec Cecho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-485) --hive-import cannot be used with --query
Date Fri, 25 May 2012 07:29:23 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283193#comment-13283193
] 

Jarek Jarcec Cecho commented on SQOOP-485:
------------------------------------------

Hi David,
thank you very much for your feedback. You're indeed right and hive do support loading differently
named files.

For you comment about apparent incompatibility between --hive-import and --query arguments.
They are fully compatible and I'm using them on daily basis. The problem is they were designed
for just one time import. You're trying to use them in sort of incremental import, which is
not supported as sqoop have explicit incremental support done with --incremental argument.

Another way that might help you easily is to make the table in HIVE partitioned and export
your daily partitions.

Jarcec
                
> --hive-import cannot be used with --query
> -----------------------------------------
>
>                 Key: SQOOP-485
>                 URL: https://issues.apache.org/jira/browse/SQOOP-485
>             Project: Sqoop
>          Issue Type: Bug
>          Components: hive-integration
>    Affects Versions: 1.3.0
>         Environment: :/home/w17626> uname -a
> Linux il93rhel91 2.6.18-128.2.1.el5 #1 SMP Wed Jul 8 11:54:47 EDT 2009 x86_64 x86_64
x86_64 GNU/Linux
>            Reporter: David Randolph
>
> sqoop import --verbose --connect jdbc:oracle:thin:@//oss-devdb2.pcs.mot.com/TOOLS --username
HADOOP --password xxx --query 'select * from fact where id < 11 and $CONDITIONS' --hive-import
--hive-table fact --fields-terminated-by , --escaped-by \\ -m1 --append
> produces the following output:
> {noformat}
> sqoop import --verbose --connect jdbc:oracle:thin:@//oss-devdb2.pcs.mot.com/TOOLS --username
HADOOP --password xxx --query 'select * from fact where id < 11 and $CONDITIONS' --hive-import
--hive-table fact --fields-terminated-by , --escaped-by \\ -m1 --append
> 12/05/15 11:10:10 DEBUG tool.BaseSqoopTool: Enabled debug logging.
> 12/05/15 11:10:10 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 12/05/15 11:10:10 DEBUG sqoop.Sqoop: Must specify destination with --target-dir.
> Try --help for usage instructions.
> Must specify destination with --target-dir.
> Try --help for usage instructions.
>         at com.cloudera.sqoop.tool.ImportTool.validateImportOptions(ImportTool.java:823)
>         at com.cloudera.sqoop.tool.ImportTool.validateOptions(ImportTool.java:890)
>         at com.cloudera.sqoop.Sqoop.run(Sqoop.java:132)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
>         at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
>         at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
>         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
> Must specify destination with --target-dir.
> Try --help for usage instructions.
> <snip>
> {noformat}
> I can work around this immediate failure by specifying the --target-dir flag. This seems
to do the right thing, but the sqoop command hangs. Also, use of --target-dir with --hive-import
is discouraged in SQOOP-464.
> How can I use the --query flag to import into Hive?
> :> sqoop version
> Sqoop 1.3.0-cdh3u3
> git commit id 57cbc8d38cc5ff22a24d34c3d13f9862fd1372bb
> Compiled by jenkins@ubuntu-slave02 on Thu Jan 26 10:41:26 PST 2012
> Thanks,
> Dave

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message