sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Douglas Spadotto <dougspado...@gmail.com>
Subject Re: Artificial timestamp column not found on --check-columns
Date Tue, 27 Oct 2015 17:19:41 GMT
Thanks Jarek. I had no idea Oracle was this limited.

Another thing I tried was to use an UDF that the DBA had to convert Epoch
to Timestamp as the --check-column value, but no luck as well. It looked
like this:

sqoop import --connect "jdbc:oracle:thin:@xxx:1521/xxx"  --username xxx
-password xxx --query 'SELECT F_MODIFICACION FROM
USU_EXT.BILHETESDEDEFEITO_BDOS WHERE $CONDITIONS" --boundary-query "select
MIN(F_MODIFICACION), MAX(F_MODIFICACION) FROM
USU_EXT.BILHETESDEDEFEITO_BDOS" --split-by F_MODIFICACION --check-column
"sir.fechas.ux2date(\"F_MODIFICACION\")" --incremental lastmodified
--last-value "2015-10-23 10:14:54.0" --target-dir
/user/p_loadbd/SIR/BILHETESDEDEFEITO_BDOS1

Yes, I can share the schema. The table has a lot of fields, so I'm going to
summarize and list only the relevant fields. They're all either VARCHAR (*)
or NUMBER (15).

Table USU_EXT.BILHETESDEDEFEITO_BDOS

ID VARCHAR2(15)
F_MODIFICACION  NUMBER(15)
...
FRA_CAUSA VARCHAR2(500)

The table is updated and appended. That is, new records are inserted, and
when a record with ID X is updated, the changed columns are updated, and
the column F_MODIFICACION is updated with the Epoch value corresponding to
the time when the change was made.

Do you have any other idea on how to work around the lack of a date or
timestamp field on the table I want to incrementally pull using Sqoop?

Thanks in advance,

Douglas


...


On Tue, Oct 27, 2015 at 9:18 AM, Jarek Jarcec Cecho <jarcec@apache.org>
wrote:

> Sqoop itself doesn’t impose any limitation on whether one can use aliases
> in the queries. That limitation is sadly exposed by Oracle itself - as it
> doesn’t allow to use aliases in “where” clause.
>
> Example:
>
> CREATE TABLE tbl(id INT);
> SELECT id AS X FROM tbl WHERE X > 10;
> [42000][904] ORA-00904: "X": invalid identifier
>
> (Run on Oracle 12.1.0.1.0)
>
> Can you share the schema with us? And also how is the table updated (are
> there new rows only appended or are some rows also changed?)
>
> Jarcec
>
> > On Oct 27, 2015, at 6:46 AM, Douglas Spadotto <dougspadotto@gmail.com>
> wrote:
> >
> > Hello everyone,
> >
> > I have a materialized view in Oracle that I need to bring into the HDFS.
> I want the load to be incremental, so I'm using the
> --check-column/--incremental/--last-value flags.
> >
> > The problem is that the view doesn't have a timestamp or date column. So
> I'm making a query generate a timestamp column.
> >
> > The command I'm using is:
> > sqoop import --connect "jdbc:oracle:thin:@<conn info>"  --username
> <user> -password <pwd> --query "SELECT F_MODIFICACION,
> TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000)
> /(60*60*24)) as MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE
> \$CONDITIONS" --boundary-query "select MIN(F_MODIFICACION),
> MAX(F_MODIFICACION) FROM USU_EXT.BILHETESDEDEFEITO_BDOS" --split-by
> "B.F_MODIFICACION" --check-column "MFDATA" --incremental lastmodified
> --last-value "2015-10-23 10:14:54.0" --target-dir
> /user/p_loadbd/SIR/BILHETESDEDEFEITO_BDOS1
> >
> > I get the following exception:
> >
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >    ...
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> > Is there a limitation on Sqoop that doesn't accept generated columns as
> values for the --check-column flag?
> >
> > (full log below)
> >
> > [p_loadbd@brtlvlts0264co ~]$ sqoop import ...
> > Warning: /usr/hdp/2.2.4.2-2/accumulo does not exist! Accumulo imports
> will fail.
> > Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> > 15/10/27 11:41:32 INFO sqoop.Sqoop: Running Sqoop version:
> 1.4.5.2.2.4.2-2
> > 15/10/27 11:41:32 WARN tool.BaseSqoopTool: Setting your password on the
> command-line is insecure. Consider using -P instead.
> > 15/10/27 11:41:33 INFO oraoop.OraOopManagerFactory: Quest® Data
> Connector for Oracle and Hadoop will not process this sqoop connection, as
> incremental mode is not supported by Quest® Data Connector for Oracle and
> Hadoop.
> > 15/10/27 11:41:33 INFO oracle.OraOopManagerFactory: Data Connector for
> Oracle and Hadoop is disabled.
> > 15/10/27 11:41:33 INFO manager.SqlManager: Using default fetchSize of
> 1000
> > 15/10/27 11:41:33 INFO tool.CodeGenTool: Beginning code generation
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> [jar:file:/usr/hdp/2.2.4.2-2/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > 15/10/27 11:41:34 INFO manager.OracleManager: Time zone has been set to
> GMT
> > 15/10/27 11:41:34 INFO manager.SqlManager: Executing SQL statement:
> SELECT F_MODIFICACION, TO_TIMESTAMP('19700101000000',
> 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as MFDATA FROM
> USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0)
> > 15/10/27 11:41:34 INFO manager.SqlManager: Executing SQL statement:
> SELECT F_MODIFICACION, TO_TIMESTAMP('19700101000000',
> 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as MFDATA FROM
> USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0)
> > 15/10/27 11:41:34 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is
> /usr/hdp/2.2.4.2-2/hadoop-mapreduce
> > Note:
> /tmp/sqoop-p_loadbd/compile/dc414065e45e677112e82a58b4bf1e68/QueryResult.java
> uses or overrides a deprecated API.
> > Note: Recompile with -Xlint:deprecation for details.
> > 15/10/27 11:41:36 INFO orm.CompilationManager: Writing jar file:
> /tmp/sqoop-p_loadbd/compile/dc414065e45e677112e82a58b4bf1e68/QueryResult.jar
> > 15/10/27 11:41:37 INFO manager.SqlManager: Executing SQL statement:
> SELECT F_MODIFICACION, TO_TIMESTAMP('19700101000000',
> 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as MFDATA FROM
> USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0)
> > 15/10/27 11:41:37 INFO tool.ImportTool: Incremental import based on
> column MFDATA
> > 15/10/27 11:41:37 INFO tool.ImportTool: Lower bound value:
> TO_TIMESTAMP('2015-10-23 10:14:54.0', 'YYYY-MM-DD HH24:MI:SS.FF')
> > 15/10/27 11:41:37 INFO tool.ImportTool: Upper bound value:
> TO_TIMESTAMP('2015-10-27 11:41:21.0', 'YYYY-MM-DD HH24:MI:SS.FF')
> > 15/10/27 11:41:37 INFO mapreduce.ImportJobBase: Beginning query import.
> > 15/10/27 11:41:37 INFO impl.TimelineClientImpl: Timeline service
> address: http://brtlvlts0264co.redecorp.br:8188/ws/v1/timeline/
> > 15/10/27 11:41:38 INFO client.ConfiguredRMFailoverProxyProvider: Failing
> over to rm2
> > 15/10/27 11:41:41 INFO db.DBInputFormat: Using read commited transaction
> isolation
> > 15/10/27 11:41:41 INFO db.DataDrivenDBInputFormat: BoundingValsQuery:
> select MIN(F_MODIFICACION), MAX(F_MODIFICACION) FROM
> USU_EXT.BILHETESDEDEFEITO_BDOS
> > 15/10/27 11:41:42 INFO mapreduce.JobSubmitter: number of splits:4
> > 15/10/27 11:41:42 INFO mapreduce.JobSubmitter: Submitting tokens for
> job: job_1445605203996_0664
> > 15/10/27 11:41:43 INFO impl.YarnClientImpl: Submitted application
> application_1445605203996_0664
> > 15/10/27 11:41:43 INFO mapreduce.Job: The url to track the job:
> http://brtlvlts0264co.redecorp.br:8088/proxy/application_1445605203996_0664/
> > 15/10/27 11:41:43 INFO mapreduce.Job: Running job: job_1445605203996_0664
> > 15/10/27 11:41:50 INFO mapreduce.Job: Job job_1445605203996_0664 running
> in uber mode : false
> > 15/10/27 11:41:50 INFO mapreduce.Job:  map 0% reduce 0%
> > 15/10/27 11:41:56 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000000_0, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:41:58 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000001_0, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:41:58 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000002_0, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:01 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000000_1, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:05 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000001_1, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:06 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000002_1, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:09 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000000_2, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:13 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000001_2, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:14 INFO mapreduce.Job: Task Id :
> attempt_1445605203996_0664_m_000002_2, Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA":
> invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at
> oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at
> oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at
> oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at
> oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at
> oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at
> org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:17 INFO mapreduce.Job:  map 100% reduce 0%
> > 15/10/27 11:42:17 INFO mapreduce.Job: Job job_1445605203996_0664 failed
> with state FAILED due to: Task failed task_1445605203996_0664_m_000000
> > Job failed as tasks failed. failedMaps:1 failedReduces:0
> >
> > 15/10/27 11:42:17 INFO mapreduce.Job: Counters: 12
> >     Job Counters
> >         Failed map tasks=10
> >         Killed map tasks=2
> >         Launched map tasks=12
> >         Other local map tasks=12
> >         Total time spent by all maps in occupied slots (ms)=111378
> >         Total time spent by all reduces in occupied slots (ms)=0
> >         Total time spent by all map tasks (ms)=55689
> >         Total vcore-seconds taken by all map tasks=55689
> >         Total megabyte-seconds taken by all map tasks=340816680
> >     Map-Reduce Framework
> >         CPU time spent (ms)=0
> >         Physical memory (bytes) snapshot=0
> >         Virtual memory (bytes) snapshot=0
> > 15/10/27 11:42:17 WARN mapreduce.Counters: Group FileSystemCounters is
> deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
> > 15/10/27 11:42:17 INFO mapreduce.ImportJobBase: Transferred 0 bytes in
> 40.1514 seconds (0 bytes/sec)
> > 15/10/27 11:42:17 INFO mapreduce.ImportJobBase: Retrieved 0 records.
> > 15/10/27 11:42:17 ERROR tool.ImportTool: Error during import: Import job
> failed!
> >
> > Thanks in advance for any assistance you may provide.
> >
> > Cheers,
> >
> > Doug
> >
> > -----
> > Frodo: "I wish none of this had happened."
> > Gandalf: "So do all who live to see such times, but that is not for them
> to decide. All we have to decide is what to do with the time that is given
> to us."
> > -- Lord of the Rings: The Fellowship of the Ring (2001)
>
>


-- 
Visite: http://canseidesercowboy.wordpress.com/
Siga: @dougspadotto ou @excowboys
-----
Frodo: "I wish none of this had happened."
Gandalf: "So do all who live to see such times, but that is not for them to
decide. All we have to decide is what to do with the time that is given to
us."
-- Lord of the Rings: The Fellowship of the Ring (2001)

Mime
View raw message