sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kuldeep Kulkarni (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3058) Sqoop import with Netezza --direct fails properly but also produces NPE
Date Fri, 24 Aug 2018 21:23:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592183#comment-16592183
] 

Kuldeep Kulkarni commented on SQOOP-3058:
-----------------------------------------

Is there any workaround for this issue? I know that it works without '--direct' however there would
be performance impact. 

 

Thanks,

Kuldeep

> Sqoop import with Netezza --direct fails properly but also produces NPE
> -----------------------------------------------------------------------
>
>                 Key: SQOOP-3058
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3058
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Markus Kemper
>            Priority: Major
>
> The [error] is expected however the [npe] seems like a defect, see [test case] below
> [error]
> ERROR:  relation does not exist SQOOP_SME_DB.SQOOP_SME1.SQOOP_SME1.T1
> [npe]
> 16/11/18 09:19:44 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
> [test case]
> {noformat}
> #################
> # STEP 01 - Setup Netezza Table and Data
> #################
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "DROP TABLE
SQOOP_SME1.T1"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "CREATE TABLE
SQOOP_SME1.T1 (C1 INTEGER)"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "INSERT INTO
SQOOP_SME1.T1 VALUES (1)"
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "SELECT C1
FROM SQOOP_SME1.T1"
> #################
> # STEP 02 - Test Import and Export (baseline)
> #################
> sqoop import --connect $MYCONN --username $MYUSER --password $MYPSWD --table "T1" --target-dir
/user/root/t1 --delete-target-dir --num-mappers 1
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "DELETE FROM
SQOOP_SME1.T1"
> sqoop export --connect $MYCONN --username $MYUSER --password $MYPSWD --table "T1" --export-dir
/user/root/t1 --num-mappers 1
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "SELECT C1
FROM SQOOP_SME1.T1"
> ---------------
> | C1          | 
> ---------------
> | 1           | 
> ---------------
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "DELETE FROM
SQOOP_SME1.T1"
> sqoop export --connect $MYCONN --username $MYUSER --password $MYPSWD --table "T1" --export-dir
/user/root/t1 --num-mappers 1 --direct
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "SELECT C1
FROM SQOOP_SME1.T1"
> ---------------
> | C1          | 
> ---------------
> | 1           | 
> ---------------
> 	
> #################
> # STEP 03 - Test Import and Export (with SCHEMA in --table option AND --direct)
> #################
> /* Notes: This failure seems correct however the NPE after the failure seems like a defect
 */
> sqoop eval --connect $MYCONN --username $MYUSER --password $MYPSWD --query "DELETE FROM
SQOOP_SME1.T1"
> sqoop export --connect $MYCONN --username $MYUSER --password $MYPSWD --table "SQOOP_SME1.T1"
--export-dir /user/root/t1 --num-mappers 1 --direct
> 16/11/18 09:19:44 ERROR manager.SqlManager: Error executing statement: org.netezza.error.NzSQLException:
ERROR:  relation does not exist SQOOP_SME_DB.SQOOP_SME1.SQOOP_SME1.T1
> org.netezza.error.NzSQLException: ERROR:  relation does not exist SQOOP_SME_DB.SQOOP_SME1.SQOOP_SME1.T1
> 	at org.netezza.internal.QueryExecutor.getNextResult(QueryExecutor.java:280)
> 	at org.netezza.internal.QueryExecutor.execute(QueryExecutor.java:76)
> 	at org.netezza.sql.NzConnection.execute(NzConnection.java:2869)
> 	at org.netezza.sql.NzPreparedStatament._execute(NzPreparedStatament.java:1126)
> 	at org.netezza.sql.NzPreparedStatament.prepare(NzPreparedStatament.java:1143)
> 	at org.netezza.sql.NzPreparedStatament.<init>(NzPreparedStatament.java:89)
> 	at org.netezza.sql.NzConnection.prepareStatement(NzConnection.java:1589)
> 	at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
> 	at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
> 	at org.apache.sqoop.manager.SqlManager.getColumnNamesForRawQuery(SqlManager.java:151)
> 	at org.apache.sqoop.manager.SqlManager.getColumnNames(SqlManager.java:116)
> 	at org.apache.sqoop.mapreduce.netezza.NetezzaExternalTableExportJob.configureOutputFormat(NetezzaExternalTableExportJob.java:128)
> 	at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:433)
> 	at org.apache.sqoop.manager.DirectNetezzaManager.exportTable(DirectNetezzaManager.java:209)
> 	at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:81)
> 	at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
> 	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> 	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> 16/11/18 09:19:44 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
> java.lang.NullPointerException
> 	at org.apache.sqoop.manager.ConnManager.escapeColNames(ConnManager.java:613)
> 	at org.apache.sqoop.mapreduce.netezza.NetezzaExternalTableExportJob.configureOutputFormat(NetezzaExternalTableExportJob.java:134)
> 	at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:433)
> 	at org.apache.sqoop.manager.DirectNetezzaManager.exportTable(DirectNetezzaManager.java:209)
> 	at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:81)
> 	at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
> 	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
> 	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
> 	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message