sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yibing Shi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2566) Sqoop incremental import using --query switch throws NullPointerException with a managed table
Date Fri, 09 Sep 2016 05:34:20 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15475986#comment-15475986
] 

Yibing Shi commented on SQOOP-2566:
-----------------------------------

Actually, should we support incrementally import (lastmodified mode) into hcatalog table?
This may cause duplicate records because the merge job would do nothing since the temp output
directory is empty. Then all the data would be *added* to hcatalog table.

> Sqoop incremental import using --query switch throws NullPointerException with a managed
table
> ----------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-2566
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2566
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.5
>            Reporter: Ajay Singh
>            Assignee: Shashank
>            Priority: Minor
>             Fix For: 1.4.7
>
>         Attachments: SQOOP-2566_0.patch
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> NullpointerException is thrown when incremental import is done with a managed table using
--query switch
> sqoop import -Dmapred.job.queue.name=default --connection-manager com.cloudera.sqoop.manager.TeradataManager
--connect <connectionString> --username lbansal --P --split-by ID --hcatalog-storage-stanza='stored
as sequencefile' --query 'select * from test where $CONDITIONS' --create-hcatalog-table --hcatalog-database
platdev --hcatalog-table test --compress --compression-codec snappy --incremental lastmodified
--check-column DAT --last-value '2014-10-17 07:00:00'
> Stacktrace 
> --------------
> 15/07/07 05:36:51 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
> java.lang.NullPointerException
> at org.apache.hadoop.fs.FileSystem.fixRelativePart(FileSystem.java:2170)
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1131)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1423)
> at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:333)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:641)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:236)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message