sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Artificial timestamp column not found on --check-columns
Date Tue, 27 Oct 2015 16:18:37 GMT
Sqoop itself doesn’t impose any limitation on whether one can use aliases in the queries.
That limitation is sadly exposed by Oracle itself - as it doesn’t allow to use aliases in
“where” clause.

Example:

CREATE TABLE tbl(id INT);
SELECT id AS X FROM tbl WHERE X > 10;
[42000][904] ORA-00904: "X": invalid identifier

(Run on Oracle 12.1.0.1.0)

Can you share the schema with us? And also how is the table updated (are there new rows only
appended or are some rows also changed?)

Jarcec

> On Oct 27, 2015, at 6:46 AM, Douglas Spadotto <dougspadotto@gmail.com> wrote:
> 
> Hello everyone,
> 
> I have a materialized view in Oracle that I need to bring into the HDFS. I want the load
to be incremental, so I'm using the --check-column/--incremental/--last-value flags.
> 
> The problem is that the view doesn't have a timestamp or date column. So I'm making a
query generate a timestamp column.
> 
> The command I'm using is:
> sqoop import --connect "jdbc:oracle:thin:@<conn info>"  --username <user>
-password <pwd> --query "SELECT F_MODIFICACION, TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000)
/(60*60*24)) as MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE \$CONDITIONS" --boundary-query
"select MIN(F_MODIFICACION), MAX(F_MODIFICACION) FROM USU_EXT.BILHETESDEDEFEITO_BDOS" --split-by
"B.F_MODIFICACION" --check-column "MFDATA" --incremental lastmodified --last-value "2015-10-23
10:14:54.0" --target-dir /user/p_loadbd/SIR/BILHETESDEDEFEITO_BDOS1
> 
> I get the following exception:
> 
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>    ...
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
> Is there a limitation on Sqoop that doesn't accept generated columns as values for the
--check-column flag?
> 
> (full log below)
> 
> [p_loadbd@brtlvlts0264co ~]$ sqoop import ...
> Warning: /usr/hdp/2.2.4.2-2/accumulo does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> 15/10/27 11:41:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.4.2-2
> 15/10/27 11:41:32 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 15/10/27 11:41:33 INFO oraoop.OraOopManagerFactory: Quest® Data Connector for Oracle
and Hadoop will not process this sqoop connection, as incremental mode is not supported by
Quest® Data Connector for Oracle and Hadoop.
> 15/10/27 11:41:33 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop
is disabled.
> 15/10/27 11:41:33 INFO manager.SqlManager: Using default fetchSize of 1000
> 15/10/27 11:41:33 INFO tool.CodeGenTool: Beginning code generation
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> 15/10/27 11:41:34 INFO manager.OracleManager: Time zone has been set to GMT
> 15/10/27 11:41:34 INFO manager.SqlManager: Executing SQL statement: SELECT F_MODIFICACION,
TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as
MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0) 
> 15/10/27 11:41:34 INFO manager.SqlManager: Executing SQL statement: SELECT F_MODIFICACION,
TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as
MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0) 
> 15/10/27 11:41:34 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.2.4.2-2/hadoop-mapreduce
> Note: /tmp/sqoop-p_loadbd/compile/dc414065e45e677112e82a58b4bf1e68/QueryResult.java uses
or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 15/10/27 11:41:36 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-p_loadbd/compile/dc414065e45e677112e82a58b4bf1e68/QueryResult.jar
> 15/10/27 11:41:37 INFO manager.SqlManager: Executing SQL statement: SELECT F_MODIFICACION,
TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as
MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0) 
> 15/10/27 11:41:37 INFO tool.ImportTool: Incremental import based on column MFDATA
> 15/10/27 11:41:37 INFO tool.ImportTool: Lower bound value: TO_TIMESTAMP('2015-10-23 10:14:54.0',
'YYYY-MM-DD HH24:MI:SS.FF')
> 15/10/27 11:41:37 INFO tool.ImportTool: Upper bound value: TO_TIMESTAMP('2015-10-27 11:41:21.0',
'YYYY-MM-DD HH24:MI:SS.FF')
> 15/10/27 11:41:37 INFO mapreduce.ImportJobBase: Beginning query import.
> 15/10/27 11:41:37 INFO impl.TimelineClientImpl: Timeline service address: http://brtlvlts0264co.redecorp.br:8188/ws/v1/timeline/
> 15/10/27 11:41:38 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to rm2
> 15/10/27 11:41:41 INFO db.DBInputFormat: Using read commited transaction isolation
> 15/10/27 11:41:41 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: select MIN(F_MODIFICACION),
MAX(F_MODIFICACION) FROM USU_EXT.BILHETESDEDEFEITO_BDOS
> 15/10/27 11:41:42 INFO mapreduce.JobSubmitter: number of splits:4
> 15/10/27 11:41:42 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1445605203996_0664
> 15/10/27 11:41:43 INFO impl.YarnClientImpl: Submitted application application_1445605203996_0664
> 15/10/27 11:41:43 INFO mapreduce.Job: The url to track the job: http://brtlvlts0264co.redecorp.br:8088/proxy/application_1445605203996_0664/
> 15/10/27 11:41:43 INFO mapreduce.Job: Running job: job_1445605203996_0664
> 15/10/27 11:41:50 INFO mapreduce.Job: Job job_1445605203996_0664 running in uber mode
: false
> 15/10/27 11:41:50 INFO mapreduce.Job:  map 0% reduce 0%
> 15/10/27 11:41:56 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000000_0,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:41:58 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000001_0,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:41:58 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000002_0,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:42:01 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000000_1,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:42:05 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000001_1,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:42:06 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000002_1,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:42:09 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000000_2,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:42:13 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000001_2,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:42:14 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000002_2,
Status : FAILED
> Error: java.io.IOException: SQLException in nextKeyValue
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
>     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
>     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
>     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> 
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
>     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
>     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
>     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
>     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
>     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
>     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
>     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
>     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
>     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
>     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
>     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
>     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
>     ... 12 more
> 
> 15/10/27 11:42:17 INFO mapreduce.Job:  map 100% reduce 0%
> 15/10/27 11:42:17 INFO mapreduce.Job: Job job_1445605203996_0664 failed with state FAILED
due to: Task failed task_1445605203996_0664_m_000000
> Job failed as tasks failed. failedMaps:1 failedReduces:0
> 
> 15/10/27 11:42:17 INFO mapreduce.Job: Counters: 12
>     Job Counters 
>         Failed map tasks=10
>         Killed map tasks=2
>         Launched map tasks=12
>         Other local map tasks=12
>         Total time spent by all maps in occupied slots (ms)=111378
>         Total time spent by all reduces in occupied slots (ms)=0
>         Total time spent by all map tasks (ms)=55689
>         Total vcore-seconds taken by all map tasks=55689
>         Total megabyte-seconds taken by all map tasks=340816680
>     Map-Reduce Framework
>         CPU time spent (ms)=0
>         Physical memory (bytes) snapshot=0
>         Virtual memory (bytes) snapshot=0
> 15/10/27 11:42:17 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use
org.apache.hadoop.mapreduce.FileSystemCounter instead
> 15/10/27 11:42:17 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 40.1514 seconds
(0 bytes/sec)
> 15/10/27 11:42:17 INFO mapreduce.ImportJobBase: Retrieved 0 records.
> 15/10/27 11:42:17 ERROR tool.ImportTool: Error during import: Import job failed!
> 
> Thanks in advance for any assistance you may provide.
> 
> Cheers,
> 
> Doug
> 
> -----
> Frodo: "I wish none of this had happened." 
> Gandalf: "So do all who live to see such times, but that is not for them to decide. All
we have to decide is what to do with the time that is given to us."
> -- Lord of the Rings: The Fellowship of the Ring (2001)


Mime
View raw message