sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: Artificial timestamp column not found on --check-columns
Date Wed, 28 Oct 2015 15:33:00 GMT
Thinking about it, another possible solution would be to create view that will add virtual
column with the UDF and import the view instead.

I’m expecting that it will incur performance penalty though :(

Jarcec

> On Oct 27, 2015, at 10:19 AM, Douglas Spadotto <dougspadotto@gmail.com> wrote:
> 
> Thanks Jarek. I had no idea Oracle was this limited.
> 
> Another thing I tried was to use an UDF that the DBA had to convert Epoch to Timestamp
as the --check-column value, but no luck as well. It looked like this:
> 
> sqoop import --connect "jdbc:oracle:thin:@xxx:1521/xxx"  --username xxx -password xxx
--query 'SELECT F_MODIFICACION FROM USU_EXT.BILHETESDEDEFEITO_BDOS WHERE $CONDITIONS" --boundary-query
"select MIN(F_MODIFICACION), MAX(F_MODIFICACION) FROM USU_EXT.BILHETESDEDEFEITO_BDOS" --split-by
F_MODIFICACION --check-column "sir.fechas.ux2date(\"F_MODIFICACION\")" --incremental lastmodified
--last-value "2015-10-23 10:14:54.0" --target-dir /user/p_loadbd/SIR/BILHETESDEDEFEITO_BDOS1
> 
> Yes, I can share the schema. The table has a lot of fields, so I'm going to summarize
and list only the relevant fields. They're all either VARCHAR (*) or NUMBER (15).
> 
> Table USU_EXT.BILHETESDEDEFEITO_BDOS
> 
> ID VARCHAR2(15)
> F_MODIFICACION  NUMBER(15)
> ...
> FRA_CAUSA VARCHAR2(500)
> 
> The table is updated and appended. That is, new records are inserted, and when a record
with ID X is updated, the changed columns are updated, and the column F_MODIFICACION is updated
with the Epoch value corresponding to the time when the change was made.
> 
> Do you have any other idea on how to work around the lack of a date or timestamp field
on the table I want to incrementally pull using Sqoop?
> 
> Thanks in advance,
> 
> Douglas
> 
> 
> ...
> 
> 
> On Tue, Oct 27, 2015 at 9:18 AM, Jarek Jarcec Cecho <jarcec@apache.org> wrote:
> Sqoop itself doesn’t impose any limitation on whether one can use aliases in the queries.
That limitation is sadly exposed by Oracle itself - as it doesn’t allow to use aliases in
“where” clause.
> 
> Example:
> 
> CREATE TABLE tbl(id INT);
> SELECT id AS X FROM tbl WHERE X > 10;
> [42000][904] ORA-00904: "X": invalid identifier
> 
> (Run on Oracle 12.1.0.1.0)
> 
> Can you share the schema with us? And also how is the table updated (are there new rows
only appended or are some rows also changed?)
> 
> Jarcec
> 
> > On Oct 27, 2015, at 6:46 AM, Douglas Spadotto <dougspadotto@gmail.com> wrote:
> >
> > Hello everyone,
> >
> > I have a materialized view in Oracle that I need to bring into the HDFS. I want
the load to be incremental, so I'm using the --check-column/--incremental/--last-value flags.
> >
> > The problem is that the view doesn't have a timestamp or date column. So I'm making
a query generate a timestamp column.
> >
> > The command I'm using is:
> > sqoop import --connect "jdbc:oracle:thin:@<conn info>"  --username <user>
-password <pwd> --query "SELECT F_MODIFICACION, TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000)
/(60*60*24)) as MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE \$CONDITIONS" --boundary-query
"select MIN(F_MODIFICACION), MAX(F_MODIFICACION) FROM USU_EXT.BILHETESDEDEFEITO_BDOS" --split-by
"B.F_MODIFICACION" --check-column "MFDATA" --incremental lastmodified --last-value "2015-10-23
10:14:54.0" --target-dir /user/p_loadbd/SIR/BILHETESDEDEFEITO_BDOS1
> >
> > I get the following exception:
> >
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >    ...
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> > Is there a limitation on Sqoop that doesn't accept generated columns as values for
the --check-column flag?
> >
> > (full log below)
> >
> > [p_loadbd@brtlvlts0264co ~]$ sqoop import ...
> > Warning: /usr/hdp/2.2.4.2-2/accumulo does not exist! Accumulo imports will fail.
> > Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> > 15/10/27 11:41:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.4.2-2
> > 15/10/27 11:41:32 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> > 15/10/27 11:41:33 INFO oraoop.OraOopManagerFactory: Quest® Data Connector for Oracle
and Hadoop will not process this sqoop connection, as incremental mode is not supported by
Quest® Data Connector for Oracle and Hadoop.
> > 15/10/27 11:41:33 INFO oracle.OraOopManagerFactory: Data Connector for Oracle and
Hadoop is disabled.
> > 15/10/27 11:41:33 INFO manager.SqlManager: Using default fetchSize of 1000
> > 15/10/27 11:41:33 INFO tool.CodeGenTool: Beginning code generation
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > 15/10/27 11:41:34 INFO manager.OracleManager: Time zone has been set to GMT
> > 15/10/27 11:41:34 INFO manager.SqlManager: Executing SQL statement: SELECT F_MODIFICACION,
TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as
MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0)
> > 15/10/27 11:41:34 INFO manager.SqlManager: Executing SQL statement: SELECT F_MODIFICACION,
TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as
MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0)
> > 15/10/27 11:41:34 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/2.2.4.2-2/hadoop-mapreduce
> > Note: /tmp/sqoop-p_loadbd/compile/dc414065e45e677112e82a58b4bf1e68/QueryResult.java
uses or overrides a deprecated API.
> > Note: Recompile with -Xlint:deprecation for details.
> > 15/10/27 11:41:36 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-p_loadbd/compile/dc414065e45e677112e82a58b4bf1e68/QueryResult.jar
> > 15/10/27 11:41:37 INFO manager.SqlManager: Executing SQL statement: SELECT F_MODIFICACION,
TO_TIMESTAMP('19700101000000', 'YYYYMMDDHH24MISS')+((F_MODIFICACION-18000) /(60*60*24)) as
MFDATA FROM USU_EXT.BILHETESDEDEFEITO_BDOS B WHERE  (1 = 0)
> > 15/10/27 11:41:37 INFO tool.ImportTool: Incremental import based on column MFDATA
> > 15/10/27 11:41:37 INFO tool.ImportTool: Lower bound value: TO_TIMESTAMP('2015-10-23
10:14:54.0', 'YYYY-MM-DD HH24:MI:SS.FF')
> > 15/10/27 11:41:37 INFO tool.ImportTool: Upper bound value: TO_TIMESTAMP('2015-10-27
11:41:21.0', 'YYYY-MM-DD HH24:MI:SS.FF')
> > 15/10/27 11:41:37 INFO mapreduce.ImportJobBase: Beginning query import.
> > 15/10/27 11:41:37 INFO impl.TimelineClientImpl: Timeline service address: http://brtlvlts0264co.redecorp.br:8188/ws/v1/timeline/
> > 15/10/27 11:41:38 INFO client.ConfiguredRMFailoverProxyProvider: Failing over to
rm2
> > 15/10/27 11:41:41 INFO db.DBInputFormat: Using read commited transaction isolation
> > 15/10/27 11:41:41 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: select MIN(F_MODIFICACION),
MAX(F_MODIFICACION) FROM USU_EXT.BILHETESDEDEFEITO_BDOS
> > 15/10/27 11:41:42 INFO mapreduce.JobSubmitter: number of splits:4
> > 15/10/27 11:41:42 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1445605203996_0664
> > 15/10/27 11:41:43 INFO impl.YarnClientImpl: Submitted application application_1445605203996_0664
> > 15/10/27 11:41:43 INFO mapreduce.Job: The url to track the job: http://brtlvlts0264co.redecorp.br:8088/proxy/application_1445605203996_0664/
> > 15/10/27 11:41:43 INFO mapreduce.Job: Running job: job_1445605203996_0664
> > 15/10/27 11:41:50 INFO mapreduce.Job: Job job_1445605203996_0664 running in uber
mode : false
> > 15/10/27 11:41:50 INFO mapreduce.Job:  map 0% reduce 0%
> > 15/10/27 11:41:56 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000000_0,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:41:58 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000001_0,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:41:58 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000002_0,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:01 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000000_1,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:05 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000001_1,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:06 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000002_1,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:09 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000000_2,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:13 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000001_2,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:14 INFO mapreduce.Job: Task Id : attempt_1445605203996_0664_m_000002_2,
Status : FAILED
> > Error: java.io.IOException: SQLException in nextKeyValue
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:277)
> >     at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> >     at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> >     at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> >     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> >     at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
> >     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> >     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:415)
> >     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> >     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> > Caused by: java.sql.SQLSyntaxErrorException: ORA-00904: "MFDATA": invalid identifier
> >
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:447)
> >     at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
> >     at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:951)
> >     at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:513)
> >     at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:227)
> >     at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
> >     at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:208)
> >     at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:886)
> >     at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:1175)
> >     at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1296)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3613)
> >     at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3657)
> >     at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1495)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:111)
> >     at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
> >     ... 12 more
> >
> > 15/10/27 11:42:17 INFO mapreduce.Job:  map 100% reduce 0%
> > 15/10/27 11:42:17 INFO mapreduce.Job: Job job_1445605203996_0664 failed with state
FAILED due to: Task failed task_1445605203996_0664_m_000000
> > Job failed as tasks failed. failedMaps:1 failedReduces:0
> >
> > 15/10/27 11:42:17 INFO mapreduce.Job: Counters: 12
> >     Job Counters
> >         Failed map tasks=10
> >         Killed map tasks=2
> >         Launched map tasks=12
> >         Other local map tasks=12
> >         Total time spent by all maps in occupied slots (ms)=111378
> >         Total time spent by all reduces in occupied slots (ms)=0
> >         Total time spent by all map tasks (ms)=55689
> >         Total vcore-seconds taken by all map tasks=55689
> >         Total megabyte-seconds taken by all map tasks=340816680
> >     Map-Reduce Framework
> >         CPU time spent (ms)=0
> >         Physical memory (bytes) snapshot=0
> >         Virtual memory (bytes) snapshot=0
> > 15/10/27 11:42:17 WARN mapreduce.Counters: Group FileSystemCounters is deprecated.
Use org.apache.hadoop.mapreduce.FileSystemCounter instead
> > 15/10/27 11:42:17 INFO mapreduce.ImportJobBase: Transferred 0 bytes in 40.1514 seconds
(0 bytes/sec)
> > 15/10/27 11:42:17 INFO mapreduce.ImportJobBase: Retrieved 0 records.
> > 15/10/27 11:42:17 ERROR tool.ImportTool: Error during import: Import job failed!
> >
> > Thanks in advance for any assistance you may provide.
> >
> > Cheers,
> >
> > Doug
> >
> > -----
> > Frodo: "I wish none of this had happened."
> > Gandalf: "So do all who live to see such times, but that is not for them to decide.
All we have to decide is what to do with the time that is given to us."
> > -- Lord of the Rings: The Fellowship of the Ring (2001)
> 
> 
> 
> 
> -- 
> Visite: http://canseidesercowboy.wordpress.com/
> Siga: @dougspadotto ou @excowboys
> -----
> Frodo: "I wish none of this had happened." 
> Gandalf: "So do all who live to see such times, but that is not for them to decide. All
we have to decide is what to do with the time that is given to us."
> -- Lord of the Rings: The Fellowship of the Ring (2001)


Mime
View raw message