sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-481) Sqoop import with --have-import using wrong column names in --columns throws a NPE
Date Wed, 02 May 2012 00:20:49 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266263#comment-13266263

Cheolsoo Park commented on SQOOP-481:

In fact, there are multiple places where the same issue can occur due to use of autoboxing.
For example, in the following lines of code in ClassWriter.generateLoadLargeObjects() can
also throw a NPE if "col" does not exist in the table because columnTypes.get() will return
a null that in turn will be autocated to a primitive integer:

int sqlType = columnTypes.get(col);
String javaType = toJavaType(col, sqlType);

Even worse, in some DBs including Oracle, it is very easy to run into this problem because
column names are case-sensitive. For example, table has a column "i" but the user may specify
"I" in an option.

What I think we should do is:
1) Identify all places where autoboxing takes place in Sqoop
2) Surround them with a try-catch block (or add a null check) and print an informative error
message such as: column 'x' does not exist in table 'y'.

Please let me know if anyone has a better suggestion.

> Sqoop import with --have-import using wrong column names in --columns throws a NPE
> ----------------------------------------------------------------------------------
>                 Key: SQOOP-481
>                 URL: https://issues.apache.org/jira/browse/SQOOP-481
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
> To reproduce the error, 
> 1) Create a table "foo" with a column name "I" on Oracle DB
> 2) Run sqoop import --connect jdbc:oracle:thin:@//localhost/xe --username **** --password
**** --verbose --table foo --split-by i --columns i --hive-import
> This generates the following call stack:
> {code}
> 12/05/01 16:12:00 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
> java.lang.NullPointerException
> 	at com.cloudera.sqoop.hive.TableDefWriter.getCreateTableStmt(TableDefWriter.java:162)
> 	at com.cloudera.sqoop.hive.HiveImport.importTable(HiveImport.java:195)
> 	at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:394)
> 	at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:455)
> 	at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> 	at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182)
> 	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221)
> 	at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230)
> 	at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
> {code}
> The reason is simple. In the following lines of code:
> {code}
> Integer colType = columnTypes.get(col);
> ...
> tring hiveColType = connManager.toHiveType(colType);
> {code}
> colType is null because column "i" does not exist in the table "foo" but "I" exists.
Now toHiveType(int colType) tries to autocast a null to a primitive int, resulting a NPE.
> It would be better if more informative message is provided rather than a random NPE.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message