sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adline Dsilva (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-2392) Failed to import from db2 with --where clause
Date Mon, 08 Jun 2015 09:40:00 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Adline Dsilva updated SQOOP-2392:
---------------------------------
    Description: 
Import from DB2 server with --where clause is failing . Issue occurs with --where clause,
full table sqoop works fine.

{code:xml}
ubuntu@node1:~$ sqoop-import --connect jdbc:db2://host:50000/GSDB --username db2inst1 --password
password -m 1 --table gosales.sales_target --where "sales_year==2004" --target-dir=/user/hive/warehouse/target123
Warning: /opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/bin/../lib/sqoop/../accumulo does
not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/06/08 16:24:44 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.4.0
15/06/08 16:24:44 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure.
Consider using -P instead.
15/06/08 16:24:44 INFO manager.SqlManager: Using default fetchSize of 1000
15/06/08 16:24:44 INFO tool.CodeGenTool: Beginning code generation
15/06/08 16:24:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
15/06/08 16:24:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
15/06/08 16:24:46 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
Note: /tmp/sqoop-ubuntu/compile/fee10be31a3243fde75ce478c2245277/gosales_sales_target.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
15/06/08 16:24:48 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-ubuntu/compile/fee10be31a3243fde75ce478c2245277/gosales.sales_target.jar
15/06/08 16:24:48 INFO mapreduce.ImportJobBase: Beginning import of gosales.sales_target
15/06/08 16:24:48 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
15/06/08 16:24:48 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
15/06/08 16:24:49 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead,
use mapreduce.job.maps
15/06/08 16:24:49 INFO client.RMProxy: Connecting to ResourceManager at node1/10.1.17.33:8032
15/06/08 16:24:54 INFO db.DBInputFormat: Using read commited transaction isolation
15/06/08 16:24:54 INFO mapreduce.JobSubmitter: number of splits:1
15/06/08 16:24:54 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1433422195897_0010
15/06/08 16:24:54 INFO impl.YarnClientImpl: Submitted application application_1433422195897_0010
15/06/08 16:24:54 INFO mapreduce.Job: The url to track the job: http://node1:8088/proxy/application_1433422195897_0010/
15/06/08 16:24:54 INFO mapreduce.Job: Running job: job_1433422195897_0010
15/06/08 16:25:00 INFO mapreduce.Job: Job job_1433422195897_0010 running in uber mode : false
15/06/08 16:25:00 INFO mapreduce.Job:  map 0% reduce 0%
15/06/08 16:25:07 INFO mapreduce.Job:  map 100% reduce 0%
15/06/08 16:25:07 INFO mapreduce.Job: Task Id : attempt_1433422195897_0010_m_000000_0, Status
: FAILED
Error: java.io.IOException: SQLException in nextKeyValue
	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
	at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-104, SQLSTATE=42601,
SQLERRMC==;WHERE ( sales_year=;<space>, DRIVER=4.19.26
	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
	at com.ibm.db2.jcc.am.fp.c(Unknown Source)
	at com.ibm.db2.jcc.am.fp.d(Unknown Source)
	at com.ibm.db2.jcc.am.fp.a(Unknown Source)
	at com.ibm.db2.jcc.am.gp.a(Unknown Source)
	at com.ibm.db2.jcc.t4.bb.h(Unknown Source)
	at com.ibm.db2.jcc.t4.bb.b(Unknown Source)
	at com.ibm.db2.jcc.t4.p.a(Unknown Source)
	at com.ibm.db2.jcc.t4.vb.i(Unknown Source)
	at com.ibm.db2.jcc.am.fp.kb(Unknown Source)
	at com.ibm.db2.jcc.am.gp.xc(Unknown Source)
	at com.ibm.db2.jcc.am.gp.b(Unknown Source)
	at com.ibm.db2.jcc.am.gp.kc(Unknown Source)
	at com.ibm.db2.jcc.am.gp.executeQuery(Unknown Source)
	at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
	... 12 more
{code}


  was:
Import from DB2 server with --where clause is failing . Issue occurs with --where clause,
full table sqoop works fine.

{code:xml}
ubuntu@node1:~$ sqoop-import --connect jdbc:db2://10.23.124.87:50000/GSDB --username db2inst1
--password password -m 1 --table gosales.sales_target --where "sales_year==2004" --target-dir=/user/hive/warehouse/target123
Warning: /opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/bin/../lib/sqoop/../accumulo does
not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/06/08 16:24:44 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.4.0
15/06/08 16:24:44 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure.
Consider using -P instead.
15/06/08 16:24:44 INFO manager.SqlManager: Using default fetchSize of 1000
15/06/08 16:24:44 INFO tool.CodeGenTool: Beginning code generation
15/06/08 16:24:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
15/06/08 16:24:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
15/06/08 16:24:46 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
Note: /tmp/sqoop-ubuntu/compile/fee10be31a3243fde75ce478c2245277/gosales_sales_target.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
15/06/08 16:24:48 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-ubuntu/compile/fee10be31a3243fde75ce478c2245277/gosales.sales_target.jar
15/06/08 16:24:48 INFO mapreduce.ImportJobBase: Beginning import of gosales.sales_target
15/06/08 16:24:48 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
15/06/08 16:24:48 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
15/06/08 16:24:49 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead,
use mapreduce.job.maps
15/06/08 16:24:49 INFO client.RMProxy: Connecting to ResourceManager at node1/10.1.17.33:8032
15/06/08 16:24:54 INFO db.DBInputFormat: Using read commited transaction isolation
15/06/08 16:24:54 INFO mapreduce.JobSubmitter: number of splits:1
15/06/08 16:24:54 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1433422195897_0010
15/06/08 16:24:54 INFO impl.YarnClientImpl: Submitted application application_1433422195897_0010
15/06/08 16:24:54 INFO mapreduce.Job: The url to track the job: http://node1:8088/proxy/application_1433422195897_0010/
15/06/08 16:24:54 INFO mapreduce.Job: Running job: job_1433422195897_0010
15/06/08 16:25:00 INFO mapreduce.Job: Job job_1433422195897_0010 running in uber mode : false
15/06/08 16:25:00 INFO mapreduce.Job:  map 0% reduce 0%
15/06/08 16:25:07 INFO mapreduce.Job:  map 100% reduce 0%
15/06/08 16:25:07 INFO mapreduce.Job: Task Id : attempt_1433422195897_0010_m_000000_0, Status
: FAILED
Error: java.io.IOException: SQLException in nextKeyValue
	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
	at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
	at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-104, SQLSTATE=42601,
SQLERRMC==;WHERE ( sales_year=;<space>, DRIVER=4.19.26
	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
	at com.ibm.db2.jcc.am.fp.c(Unknown Source)
	at com.ibm.db2.jcc.am.fp.d(Unknown Source)
	at com.ibm.db2.jcc.am.fp.a(Unknown Source)
	at com.ibm.db2.jcc.am.gp.a(Unknown Source)
	at com.ibm.db2.jcc.t4.bb.h(Unknown Source)
	at com.ibm.db2.jcc.t4.bb.b(Unknown Source)
	at com.ibm.db2.jcc.t4.p.a(Unknown Source)
	at com.ibm.db2.jcc.t4.vb.i(Unknown Source)
	at com.ibm.db2.jcc.am.fp.kb(Unknown Source)
	at com.ibm.db2.jcc.am.gp.xc(Unknown Source)
	at com.ibm.db2.jcc.am.gp.b(Unknown Source)
	at com.ibm.db2.jcc.am.gp.kc(Unknown Source)
	at com.ibm.db2.jcc.am.gp.executeQuery(Unknown Source)
	at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
	... 12 more
{code}



> Failed to import from db2 with --where clause
> ---------------------------------------------
>
>                 Key: SQOOP-2392
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2392
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors/generic
>    Affects Versions: 1.4.5
>         Environment: Ubuntu 12.04/DB2 CDH-5.4
> Sqoop 1.4.5-cdh5.4.0
>            Reporter: Adline Dsilva
>
> Import from DB2 server with --where clause is failing . Issue occurs with --where clause,
full table sqoop works fine.
> {code:xml}
> ubuntu@node1:~$ sqoop-import --connect jdbc:db2://host:50000/GSDB --username db2inst1
--password password -m 1 --table gosales.sales_target --where "sales_year==2004" --target-dir=/user/hive/warehouse/target123
> Warning: /opt/cloudera/parcels/CDH-5.4.0-1.cdh5.4.0.p0.27/bin/../lib/sqoop/../accumulo
does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> 15/06/08 16:24:44 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.4.0
> 15/06/08 16:24:44 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 15/06/08 16:24:44 INFO manager.SqlManager: Using default fetchSize of 1000
> 15/06/08 16:24:44 INFO tool.CodeGenTool: Beginning code generation
> 15/06/08 16:24:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
> 15/06/08 16:24:46 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
> 15/06/08 16:24:46 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce
> Note: /tmp/sqoop-ubuntu/compile/fee10be31a3243fde75ce478c2245277/gosales_sales_target.java
uses or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 15/06/08 16:24:48 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-ubuntu/compile/fee10be31a3243fde75ce478c2245277/gosales.sales_target.jar
> 15/06/08 16:24:48 INFO mapreduce.ImportJobBase: Beginning import of gosales.sales_target
> 15/06/08 16:24:48 INFO Configuration.deprecation: mapred.jar is deprecated. Instead,
use mapreduce.job.jar
> 15/06/08 16:24:48 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM gosales.sales_target
AS t WHERE 1=0
> 15/06/08 16:24:49 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead,
use mapreduce.job.maps
> 15/06/08 16:24:49 INFO client.RMProxy: Connecting to ResourceManager at node1/10.1.17.33:8032
> 15/06/08 16:24:54 INFO db.DBInputFormat: Using read commited transaction isolation
> 15/06/08 16:24:54 INFO mapreduce.JobSubmitter: number of splits:1
> 15/06/08 16:24:54 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1433422195897_0010
> 15/06/08 16:24:54 INFO impl.YarnClientImpl: Submitted application application_1433422195897_0010
> 15/06/08 16:24:54 INFO mapreduce.Job: The url to track the job: http://node1:8088/proxy/application_1433422195897_0010/
> 15/06/08 16:24:54 INFO mapreduce.Job: Running job: job_1433422195897_0010
> 15/06/08 16:25:00 INFO mapreduce.Job: Job job_1433422195897_0010 running in uber mode
: false
> 15/06/08 16:25:00 INFO mapreduce.Job:  map 0% reduce 0%
> 15/06/08 16:25:07 INFO mapreduce.Job:  map 100% reduce 0%
> 15/06/08 16:25:07 INFO mapreduce.Job: Task Id : attempt_1433422195897_0010_m_000000_0,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
> 	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> 	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
> 	at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> 	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> 	at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-104, SQLSTATE=42601,
SQLERRMC==;WHERE ( sales_year=;<space>, DRIVER=4.19.26
> 	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
> 	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
> 	at com.ibm.db2.jcc.am.kd.a(Unknown Source)
> 	at com.ibm.db2.jcc.am.fp.c(Unknown Source)
> 	at com.ibm.db2.jcc.am.fp.d(Unknown Source)
> 	at com.ibm.db2.jcc.am.fp.a(Unknown Source)
> 	at com.ibm.db2.jcc.am.gp.a(Unknown Source)
> 	at com.ibm.db2.jcc.t4.bb.h(Unknown Source)
> 	at com.ibm.db2.jcc.t4.bb.b(Unknown Source)
> 	at com.ibm.db2.jcc.t4.p.a(Unknown Source)
> 	at com.ibm.db2.jcc.t4.vb.i(Unknown Source)
> 	at com.ibm.db2.jcc.am.fp.kb(Unknown Source)
> 	at com.ibm.db2.jcc.am.gp.xc(Unknown Source)
> 	at com.ibm.db2.jcc.am.gp.b(Unknown Source)
> 	at com.ibm.db2.jcc.am.gp.kc(Unknown Source)
> 	at com.ibm.db2.jcc.am.gp.executeQuery(Unknown Source)
> 	at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> 	at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> 	... 12 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message