sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leonardo Brambilla <lbrambi...@contractor.elance-odesk.com>
Subject Re: Fwd: Sqoop export not working when using "update-key"
Date Tue, 29 Jul 2014 18:37:51 GMT
Hello there, does anybody here use Sqoop to export and update to Oracle DB?
can you please share the command you use?
thanks


On Mon, Jul 21, 2014 at 10:40 AM, Leonardo Brambilla <
lbrambilla@contractor.elance-odesk.com> wrote:

> Hello, any thoughts on this?
> Can someone tell me if possible to run sqoop export and specify a jar file
> and the main class? I tried "--jar-file" and "--class-name" without luck.
> Does anyone have an example of a running command using jar and class?
> Thanks
>
>
> On Fri, Jul 18, 2014 at 1:59 PM, Leonardo Brambilla <
> lbrambilla@contractor.elance-odesk.com> wrote:
>
>> Yes, I'm using all upper case, and you can see in the log extract that
>> the query built is:
>> 14/07/18 12:45:04 INFO manager.SqlManager: Executing SQL statement:
>> SELECT t.* FROM SEARCH_KEYWORDS_AGGREGATION t WHERE 1=0
>>
>> which is fine, and to be sure I copy pasted it into sqldeveloper and it
>> runs ok.
>> Maybe Sqoop is having problems parsing the result from oracle in my
>> environment, not sure.
>>
>> thanks
>>
>>
>> On Fri, Jul 18, 2014 at 1:53 PM, Gwen Shapira <gshapira@cloudera.com>
>> wrote:
>>
>>> Looks like Sqoop is failing to fetch column list from the table. Can
>>> you check that $table is ALL UPPER CASE?
>>> We are doing "select ... from user_tab_columns where table_name='$table';
>>>
>>> Oracle stores tables as upper case, so $table has to match.
>>>
>>> Gwen
>>>
>>> On Fri, Jul 18, 2014 at 9:49 AM, Leonardo Brambilla
>>> <lbrambilla@contractor.elance-odesk.com> wrote:
>>> > Well, if I try to run the command without --columns I get an error:
>>> > Attempted to generate class with no columns!
>>> > Part of the log:
>>> > 14/07/18 12:45:04 INFO tool.CodeGenTool: Beginning code generation
>>> > 14/07/18 12:45:04 DEBUG manager.OracleManager: Using column names
>>> query:
>>> > SELECT t.* FROM SEARCH_KEYWORDS_AGGREGATION t WHERE 1=0
>>> > 14/07/18 12:45:04 DEBUG manager.SqlManager: Execute
>>> getColumnTypesRawQuery :
>>> > SELECT t.* FROM SEARCH_KEYWORDS_AGGREGATION t WHERE 1=0
>>> > 14/07/18 12:45:04 DEBUG manager.OracleManager$ConnCache: Got cached
>>> > connection for jdbc:oracle:thin:@devbox.com:1541/devdb/uDev
>>> > 14/07/18 12:45:04 INFO manager.OracleManager: Time zone has been set
>>> to GMT
>>> > 14/07/18 12:45:04 DEBUG manager.SqlManager: Using fetchSize for next
>>> query:
>>> > 1000
>>> > 14/07/18 12:45:04 INFO manager.SqlManager: Executing SQL statement:
>>> SELECT
>>> > t.* FROM SEARCH_KEYWORDS_AGGREGATION t WHERE 1=0
>>> > 14/07/18 12:45:04 DEBUG manager.OracleManager$ConnCache: Caching
>>> released
>>> > connection for jdbc:oracle:thin:@devbox.com:1541/devdb/uDev
>>> > 14/07/18 12:45:04 DEBUG orm.ClassWriter: selected columns:
>>> > 14/07/18 12:45:04 DEBUG orm.ClassWriter: db write column order:
>>> > 14/07/18 12:45:04 DEBUG orm.ClassWriter:   SEARCH_DATE [from
>>> --update-key
>>> > parameter]
>>> > 14/07/18 12:45:04 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>> > java.lang.IllegalArgumentException: Attempted to generate class with no
>>> > columns!
>>> > java.lang.IllegalArgumentException: Attempted to generate class with no
>>> > columns!
>>> >         at
>>> >
>>> org.apache.sqoop.orm.ClassWriter.generateClassForColumns(ClassWriter.java:1295)
>>> >         at
>>> org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1176)
>>> >         at
>>> > org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:96)
>>> >         at
>>> org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
>>> >         at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
>>> >         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>>> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> >         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>>> >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>>> >         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>>> >         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>>> >
>>> > look in the log there is a "selected columns: " (empty)
>>> >
>>> >
>>> > On Fri, Jul 18, 2014 at 1:13 PM, Gwen Shapira <gshapira@cloudera.com>
>>> wrote:
>>> >>
>>> >> If we omit --columns the source file has to match the DB in number of
>>> >> columns, types and order.
>>> >>
>>> >> On Fri, Jul 18, 2014 at 8:23 AM, Leonardo Brambilla
>>> >> <lbrambilla@contractor.elance-odesk.com> wrote:
>>> >> > Hello some more findings, could this bug be related to my problem?
>>> >> > https://issues.apache.org/jira/browse/SQOOP-824
>>> >> > I know it says it's fixed since 1.4.3 but maybe that brought some
>>> other
>>> >> > case.
>>> >> > I still don't understand how to run export command without
>>> specifying
>>> >> > the
>>> >> > --columns parameter, can you tell me what is the default behavior
>>> when
>>> >> > you
>>> >> > omit --columns? Does the source file need to have the same column
>>> order
>>> >> > than
>>> >> > the target table?
>>> >> >
>>> >> > Thanks
>>> >> >
>>> >> >
>>> >> > On Fri, Jul 18, 2014 at 12:46 AM, Leonardo Brambilla
>>> >> > <lbrambilla@contractor.elance-odesk.com> wrote:
>>> >> >>
>>> >> >> I think I found something. The java class generated when using
>>> >> >> update-key
>>> >> >> differs from the one without update-key. The one that throws
>>> exception
>>> >> >> is
>>> >> >> missing to write the fields that are not specified in the
>>> update-key. I
>>> >> >> also
>>> >> >> see (with --verbose) that when using --update-key there is an extra
>>> >> >> debug
>>> >> >> line that says
>>> >> >> 14/07/17 22:53:27 DEBUG orm.ClassWriter: db write column order:
>>> >> >> 14/07/17 22:53:27 DEBUG orm.ClassWriter:   SEARCH_DATE
>>> >> >>
>>> >> >> Below is the method generated for the command without --update-key
>>> >> >>
>>> >> >>   public int write(PreparedStatement __dbStmt, int __off) throws
>>> >> >> SQLException {
>>> >> >>     JdbcWritableBridge.writeTimestamp(SEARCH_DATE, 1 + __off, 93,
>>> >> >> __dbStmt);
>>> >> >>     JdbcWritableBridge.writeString(SEARCH_TYPE, 2 + __off, 12,
>>> >> >> __dbStmt);
>>> >> >>     JdbcWritableBridge.writeString(USER_AGENT, 3 + __off, 12,
>>> >> >> __dbStmt);
>>> >> >>     JdbcWritableBridge.writeString(SRCH_KEYWORD, 4 + __off, 12,
>>> >> >> __dbStmt);
>>> >> >>     JdbcWritableBridge.writeBigDecimal(SRCH_COUNT, 5 + __off, 2,
>>> >> >> __dbStmt);
>>> >> >>     return 5;
>>> >> >>   }
>>> >> >>
>>> >> >> Below is the one generated for the command with --update-key
>>> >> >>   public int write(PreparedStatement __dbStmt, int __off) throws
>>> >> >> SQLException {
>>> >> >>     JdbcWritableBridge.writeTimestamp(SEARCH_DATE, 1 + __off, 93,
>>> >> >> __dbStmt);
>>> >> >>     return 1;
>>> >> >>   }
>>> >> >>
>>> >> >> I tried to force export to use the properly generated class with
>>> >> >> parameters "jar-file" and "class-name" but that didn't work, if
>>> like
>>> >> >> those
>>> >> >> params are not allowed in the export command. This is what I tried
>>> to
>>> >> >> force
>>> >> >> using the properly generated source
>>> >> >> sqoop export \
>>> >> >> --connect jdbc:oracle:thin:@ddb04.local.com:1541/test04 \
>>> >> >> --update-key "SEARCH_DATE" \
>>> >> >> --columns $columns \
>>> >> >> --table $table --username $user --password $passwd \
>>> >> >> --fields-terminated-by "=" --export-dir $exportDir
>>> >> >> --jar-file SEARCH_TABLE.jar --class-name SEARCH_TABLE
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> On Thu, Jul 17, 2014 at 5:04 PM, Leonardo Brambilla
>>> >> >> <lbrambilla@contractor.elance-odesk.com> wrote:
>>> >> >>>
>>> >> >>> Yes, the update-key is a subset of columns.
>>> >> >>>
>>> >> >>>
>>> >> >>> On Thu, Jul 17, 2014 at 4:16 PM, Gwen Shapira <
>>> gshapira@cloudera.com>
>>> >> >>> wrote:
>>> >> >>>>
>>> >> >>>> Does the update column appear in $columns? It should be in there.
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> On Thu, Jul 17, 2014 at 10:48 AM, Leonardo Brambilla
>>> >> >>>> <lbrambilla@contractor.elance-odesk.com> wrote:
>>> >> >>>>>
>>> >> >>>>> Hi Gwen, thank you for replying.
>>> >> >>>>>
>>> >> >>>>> I went to the data node, the userlogs and all I found in syslog
>>> file
>>> >> >>>>> is
>>> >> >>>>> what I already posted:
>>> >> >>>>> 2014-07-17 10:19:09,280 INFO
>>> >> >>>>> org.apache.hadoop.util.NativeCodeLoader:
>>> >> >>>>> Loaded the native-hadoop library
>>> >> >>>>> 2014-07-17 10:19:09,700 INFO org.apache.hadoop.util.ProcessTree:
>>> >> >>>>> setsid
>>> >> >>>>> exited with exit code 0
>>> >> >>>>> 2014-07-17 10:19:09,706 INFO org.apache.hadoop.mapred.Task:
>>>  Using
>>> >> >>>>> ResourceCalculatorPlugin :
>>> >> >>>>> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@34c3a7c0
>>> >> >>>>> 2014-07-17 10:19:10,266 INFO
>>> >> >>>>> org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress
>>> thread
>>> >> >>>>> is
>>> >> >>>>> finished. keepGoing=false
>>> >> >>>>> 2014-07-17 10:19:10,476 INFO
>>> >> >>>>> org.apache.hadoop.mapred.TaskLogsTruncater: Initializing logs'
>>> >> >>>>> truncater
>>> >> >>>>> with mapRetainSize=-1 and reduceRetainSize=-1
>>> >> >>>>> 2014-07-17 10:19:10,537 INFO
>>> org.apache.hadoop.io.nativeio.NativeIO:
>>> >> >>>>> Initialized cache for UID to User mapping with a cache timeout
>>> of
>>> >> >>>>> 14400
>>> >> >>>>> seconds.
>>> >> >>>>> 2014-07-17 10:19:10,537 INFO
>>> org.apache.hadoop.io.nativeio.NativeIO:
>>> >> >>>>> Got UserName elance for UID 666 from the native implementation
>>> >> >>>>> 2014-07-17 10:19:10,539 ERROR
>>> >> >>>>> org.apache.hadoop.security.UserGroupInformation:
>>> >> >>>>> PriviledgedActionException
>>> >> >>>>> as:elance cause:java.io.IOException: java.sql.SQLException:
>>> Missing
>>> >> >>>>> IN or
>>> >> >>>>> OUT parameter at index:: 2
>>> >> >>>>> 2014-07-17 10:19:10,540 WARN org.apache.hadoop.mapred.Child:
>>> Error
>>> >> >>>>> running child
>>> >> >>>>> java.io.IOException: java.sql.SQLException: Missing IN or OUT
>>> >> >>>>> parameter
>>> >> >>>>> at index:: 2
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:651)
>>> >> >>>>> at
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
>>> >> >>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>>> >> >>>>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>> >> >>>>> at java.security.AccessController.doPrivileged(Native Method)
>>> >> >>>>> at javax.security.auth.Subject.doAs(Subject.java:415)
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>>> >> >>>>> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>>> >> >>>>> Caused by: java.sql.SQLException: Missing IN or OUT parameter at
>>> >> >>>>> index:: 2
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> oracle.jdbc.driver.OraclePreparedStatement.processCompletedBindRow(OraclePreparedStatement.java:1844)
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> oracle.jdbc.driver.OraclePreparedStatement.addBatch(OraclePreparedStatement.java:10213)
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> oracle.jdbc.driver.OraclePreparedStatementWrapper.addBatch(OraclePreparedStatementWrapper.java:1362)
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> org.apache.sqoop.mapreduce.UpdateOutputFormat$UpdateRecordWriter.getPreparedStatement(UpdateOutputFormat.java:174)
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> >> >>>>> at
>>> >> >>>>>
>>> >> >>>>>
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>> >> >>>>> ... 8 more
>>> >> >>>>> 2014-07-17 10:19:10,543 INFO org.apache.hadoop.mapred.Task:
>>> Runnning
>>> >> >>>>> cleanup for the task
>>> >> >>>>>
>>> >> >>>>> There isn't more data than that.
>>> >> >>>>> Can you please check my sqoop command and validate that I'm
>>> using
>>> >> >>>>> the
>>> >> >>>>> proper arguments? The argument "--columns" is used in export to
>>> tell
>>> >> >>>>> sqoop
>>> >> >>>>> the order in which it should read the columns from the file
>>> right?
>>> >> >>>>> Does the last column need to have delimiter too?
>>> >> >>>>> The source file should be ok, have in mind that it works for
>>> insert
>>> >> >>>>> but
>>> >> >>>>> fails when I add the parameter --update-key
>>> >> >>>>>
>>> >> >>>>> Thanks
>>> >> >>>>> Leo
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>> On Thu, Jul 17, 2014 at 1:52 PM, Gwen Shapira
>>> >> >>>>> <gshapira@cloudera.com>
>>> >> >>>>> wrote:
>>> >> >>>>>>
>>> >> >>>>>> I can confirm that Sqoop export update works on Oracle, both
>>> with
>>> >> >>>>>> and
>>> >> >>>>>> without Oraoop.
>>> >> >>>>>>
>>> >> >>>>>> The specific exception you are getting indicates that Oracle
>>> >> >>>>>> expects
>>> >> >>>>>> at least 4 columns of data and the HDFS file may have less than
>>> >> >>>>>> that.
>>> >> >>>>>>
>>> >> >>>>>> Can you double check that the columns in Oracle and your data
>>> file
>>> >> >>>>>> match? And that you are using a correct delimiter?
>>> >> >>>>>>
>>> >> >>>>>> And as Jarcec said, if you have access to the Task Tracker user
>>> >> >>>>>> logs
>>> >> >>>>>> for one of the mappers, you'll have much more details to work
>>> with
>>> >> >>>>>> - for
>>> >> >>>>>> example the specific line that failed.
>>> >> >>>>>>
>>> >> >>>>>> Gwen
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>> On Thu, Jul 17, 2014 at 7:44 AM, Leonardo Brambilla
>>> >> >>>>>> <lbrambilla@contractor.elance-odesk.com> wrote:
>>> >> >>>>>>>
>>> >> >>>>>>> Hello Jarek,
>>> >> >>>>>>>
>>> >> >>>>>>> I'm getting back to this issue, I'm trying to fix it by using
>>> >> >>>>>>> Oraoop
>>> >> >>>>>>> but that doesn't avoid the exception:
>>> >> >>>>>>> java.io.IOException: java.sql.SQLException: Missing IN or OUT
>>> >> >>>>>>> parameter at index:: 4
>>> >> >>>>>>>
>>> >> >>>>>>> I ran a couple of tests and I can tell that the following
>>> command
>>> >> >>>>>>> works to insert new rows:
>>> >> >>>>>>> sqoop export \
>>> >> >>>>>>> --connect jdbc:oracle:thin:@ddb04.local.com:1541/test04 \
>>> >> >>>>>>> --columns $columns \
>>> >> >>>>>>> --table $table --username $user --password $passwd \
>>> >> >>>>>>> --fields-terminated-by "=" --export-dir $exportDir
>>> >> >>>>>>>
>>> >> >>>>>>> But the following command (just added --update-key) throws an
>>> >> >>>>>>> exception:
>>> >> >>>>>>> sqoop export \
>>> >> >>>>>>> --connect jdbc:oracle:thin:@ddb04.local.com:1541/test04 \
>>> >> >>>>>>> --update-key "SEARCH_DATE" \
>>> >> >>>>>>> --columns $columns \
>>> >> >>>>>>> --table $table --username $user --password $passwd \
>>> >> >>>>>>> --fields-terminated-by "=" --export-dir $exportDir
>>> >> >>>>>>>
>>> >> >>>>>>> DB is oracle 11.2.0.2.0
>>> >> >>>>>>> Sqoop is 1.4.4
>>> >> >>>>>>> Java 1.7
>>> >> >>>>>>> Oraoop 1.6
>>> >> >>>>>>> Oracle jdbc driver "ojdb6c.jar" implementation version
>>> 11.2.0.3.0
>>> >> >>>>>>>
>>> >> >>>>>>> Like I said before, all the log I can get from the failed
>>> task I
>>> >> >>>>>>> already posted here.
>>> >> >>>>>>>
>>> >> >>>>>>> Can you confirm that Sqoop export update works on Oracle DBs?
>>> >> >>>>>>> Thanks in advance
>>> >> >>>>>>> Leo
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>> On Fri, May 16, 2014 at 4:51 PM, Jarek Jarcec Cecho
>>> >> >>>>>>> <jarcec@apache.org> wrote:
>>> >> >>>>>>>>
>>> >> >>>>>>>> Hi Leonardo,
>>> >> >>>>>>>> sadly the Sqoop output might not be that much helpful in this
>>> >> >>>>>>>> case,
>>> >> >>>>>>>> could you please share with us the failed map task log?
>>> >> >>>>>>>>
>>> >> >>>>>>>> The easiest way how to get it on Hadoop 1.x is to open the
>>> job
>>> >> >>>>>>>> tracker webinterface, find the failed Sqoop job and navigate
>>> to
>>> >> >>>>>>>> the failed
>>> >> >>>>>>>> map tasks.
>>> >> >>>>>>>>
>>> >> >>>>>>>> Jarcec
>>> >> >>>>>>>>
>>> >> >>>>>>>> On Tue, May 13, 2014 at 11:36:34AM -0300, Leonardo Brambilla
>>> >> >>>>>>>> wrote:
>>> >> >>>>>>>> > Hi Jarek, find below the full sqoop generated log. I went
>>> >> >>>>>>>> > through
>>> >> >>>>>>>> > all the
>>> >> >>>>>>>> > Cluster's nodes for this task logs and there is nothing
>>> more
>>> >> >>>>>>>> > than
>>> >> >>>>>>>> > this same
>>> >> >>>>>>>> > error. I really don't know what else to look for.
>>> >> >>>>>>>> >
>>> >> >>>>>>>> > Thanks
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> > Warning: /usr/lib/hbase does not exist! HBase imports will
>>> >> >>>>>>>> > fail.
>>> >> >>>>>>>> > Please set $HBASE_HOME to the root of your HBase
>>> installation.
>>> >> >>>>>>>> > 14/05/13 10:26:41 WARN tool.BaseSqoopTool: Setting your
>>> >> >>>>>>>> > password
>>> >> >>>>>>>> > on the
>>> >> >>>>>>>> > command-line is insecure. Consider using -P instead.
>>> >> >>>>>>>> > 14/05/13 10:26:41 INFO manager.SqlManager: Using default
>>> >> >>>>>>>> > fetchSize
>>> >> >>>>>>>> > of 1000
>>> >> >>>>>>>> > 14/05/13 10:26:41 INFO manager.OracleManager: Time zone has
>>> >> >>>>>>>> > been
>>> >> >>>>>>>> > set to GMT
>>> >> >>>>>>>> > 14/05/13 10:26:41 INFO tool.CodeGenTool: Beginning code
>>> >> >>>>>>>> > generation
>>> >> >>>>>>>> > 14/05/13 10:26:41 INFO manager.OracleManager: Time zone has
>>> >> >>>>>>>> > been
>>> >> >>>>>>>> > set to GMT
>>> >> >>>>>>>> > 14/05/13 10:26:41 INFO manager.SqlManager: Executing SQL
>>> >> >>>>>>>> > statement: SELECT
>>> >> >>>>>>>> > t.* FROM etl.EXPT_SPAM_RED_JOB t WHERE 1=0
>>> >> >>>>>>>> > 14/05/13 10:26:41 INFO orm.CompilationManager:
>>> >> >>>>>>>> > HADOOP_MAPRED_HOME
>>> >> >>>>>>>> > is
>>> >> >>>>>>>> > /home/elance/hadoop
>>> >> >>>>>>>> > Note:
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> /tmp/sqoop-elance/compile/9f8f413ab105fbe67d985bdb29534d27/etl_EXPT_SPAM_RED_JOB.java
>>> >> >>>>>>>> > uses or overrides a deprecated API.
>>> >> >>>>>>>> > Note: Recompile with -Xlint:deprecation for details.
>>> >> >>>>>>>> > 14/05/13 10:26:42 INFO orm.CompilationManager: Writing jar
>>> >> >>>>>>>> > file:
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> /tmp/sqoop-elance/compile/9f8f413ab105fbe67d985bdb29534d27/etl.EXPT_SPAM_RED_JOB.jar
>>> >> >>>>>>>> > 14/05/13 10:26:42 INFO mapreduce.ExportJobBase: Beginning
>>> >> >>>>>>>> > export
>>> >> >>>>>>>> > of
>>> >> >>>>>>>> > etl.EXPT_SPAM_RED_JOB
>>> >> >>>>>>>> > 14/05/13 10:26:43 INFO input.FileInputFormat: Total input
>>> paths
>>> >> >>>>>>>> > to
>>> >> >>>>>>>> > process
>>> >> >>>>>>>> > : 1
>>> >> >>>>>>>> > 14/05/13 10:26:43 INFO input.FileInputFormat: Total input
>>> paths
>>> >> >>>>>>>> > to
>>> >> >>>>>>>> > process
>>> >> >>>>>>>> > : 1
>>> >> >>>>>>>> > 14/05/13 10:26:44 INFO mapred.JobClient: Running job:
>>> >> >>>>>>>> > job_201404190827_0998
>>> >> >>>>>>>> > 14/05/13 10:26:45 INFO mapred.JobClient:  map 0% reduce 0%
>>> >> >>>>>>>> > 14/05/13 10:26:53 INFO mapred.JobClient:  map 25% reduce 0%
>>> >> >>>>>>>> > 14/05/13 10:26:54 INFO mapred.JobClient:  map 75% reduce 0%
>>> >> >>>>>>>> > 14/05/13 10:26:55 INFO mapred.JobClient: Task Id :
>>> >> >>>>>>>> > attempt_201404190827_0998_m_000001_0, Status : FAILED
>>> >> >>>>>>>> > java.io.IOException: java.sql.SQLException: Missing IN or
>>> OUT
>>> >> >>>>>>>> > parameter at
>>> >> >>>>>>>> > index:: 4
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:651)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> > org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>>> >> >>>>>>>> >         at
>>> org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>> >> >>>>>>>> >         at
>>> java.security.AccessController.doPrivileged(Native
>>> >> >>>>>>>> > Method)
>>> >> >>>>>>>> >         at
>>> javax.security.auth.Subject.doAs(Subject.java:415)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>>> >> >>>>>>>> >         at
>>> org.apache.hadoop.mapred.Child.main(Child.java:249)
>>> >> >>>>>>>> > Caused by: java.sql.SQLException: Missing IN or OUT
>>> parameter
>>> >> >>>>>>>> > at
>>> >> >>>>>>>> > index:: 4
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatement.processCompletedBindRow(OraclePreparedStatement.java:1844)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatement.addBatch(OraclePreparedStatement.java:10213)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatementWrapper.addBatch(OraclePreparedStatementWrapper.java:1362)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.UpdateOutputFormat$UpdateRecordWriter.getPreparedStatement(UpdateOutputFormat.java:174)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>> >> >>>>>>>> >         ... 8 more
>>> >> >>>>>>>> >
>>> >> >>>>>>>> > 14/05/13 10:27:00 INFO mapred.JobClient: Task Id :
>>> >> >>>>>>>> > attempt_201404190827_0998_m_000001_1, Status : FAILED
>>> >> >>>>>>>> > java.io.IOException: java.sql.SQLException: Missing IN or
>>> OUT
>>> >> >>>>>>>> > parameter at
>>> >> >>>>>>>> > index:: 4
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:651)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> > org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>>> >> >>>>>>>> >         at
>>> org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>> >> >>>>>>>> >         at
>>> java.security.AccessController.doPrivileged(Native
>>> >> >>>>>>>> > Method)
>>> >> >>>>>>>> >         at
>>> javax.security.auth.Subject.doAs(Subject.java:415)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>>> >> >>>>>>>> >         at
>>> org.apache.hadoop.mapred.Child.main(Child.java:249)
>>> >> >>>>>>>> > Caused by: java.sql.SQLException: Missing IN or OUT
>>> parameter
>>> >> >>>>>>>> > at
>>> >> >>>>>>>> > index:: 4
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatement.processCompletedBindRow(OraclePreparedStatement.java:1844)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatement.addBatch(OraclePreparedStatement.java:10213)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatementWrapper.addBatch(OraclePreparedStatementWrapper.java:1362)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.UpdateOutputFormat$UpdateRecordWriter.getPreparedStatement(UpdateOutputFormat.java:174)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>> >> >>>>>>>> >         ... 8 more
>>> >> >>>>>>>> >
>>> >> >>>>>>>> > 14/05/13 10:27:05 INFO mapred.JobClient: Task Id :
>>> >> >>>>>>>> > attempt_201404190827_0998_m_000001_2, Status : FAILED
>>> >> >>>>>>>> > java.io.IOException: java.sql.SQLException: Missing IN or
>>> OUT
>>> >> >>>>>>>> > parameter at
>>> >> >>>>>>>> > index:: 4
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:184)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:651)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> > org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>>> >> >>>>>>>> >         at
>>> org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>> >> >>>>>>>> >         at
>>> java.security.AccessController.doPrivileged(Native
>>> >> >>>>>>>> > Method)
>>> >> >>>>>>>> >         at
>>> javax.security.auth.Subject.doAs(Subject.java:415)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>>> >> >>>>>>>> >         at
>>> org.apache.hadoop.mapred.Child.main(Child.java:249)
>>> >> >>>>>>>> > Caused by: java.sql.SQLException: Missing IN or OUT
>>> parameter
>>> >> >>>>>>>> > at
>>> >> >>>>>>>> > index:: 4
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatement.processCompletedBindRow(OraclePreparedStatement.java:1844)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatement.addBatch(OraclePreparedStatement.java:10213)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> oracle.jdbc.driver.OraclePreparedStatementWrapper.addBatch(OraclePreparedStatementWrapper.java:1362)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.UpdateOutputFormat$UpdateRecordWriter.getPreparedStatement(UpdateOutputFormat.java:174)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> >> >>>>>>>> >         at
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(AsyncSqlRecordWriter.java:181)
>>> >> >>>>>>>> >         ... 8 more
>>> >> >>>>>>>> >
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient: Job complete:
>>> >> >>>>>>>> > job_201404190827_0998
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient: Counters: 20
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:   Job Counters
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:
>>> >> >>>>>>>> > SLOTS_MILLIS_MAPS=30548
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Total time
>>> spent
>>> >> >>>>>>>> > by
>>> >> >>>>>>>> > all
>>> >> >>>>>>>> > reduces waiting after reserving slots (ms)=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Total time
>>> spent
>>> >> >>>>>>>> > by
>>> >> >>>>>>>> > all maps
>>> >> >>>>>>>> > waiting after reserving slots (ms)=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Rack-local map
>>> >> >>>>>>>> > tasks=5
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Launched map
>>> >> >>>>>>>> > tasks=7
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Data-local map
>>> >> >>>>>>>> > tasks=2
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:
>>> >> >>>>>>>> > SLOTS_MILLIS_REDUCES=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Failed map
>>> tasks=1
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:   File Output
>>> Format
>>> >> >>>>>>>> > Counters
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Bytes
>>> Written=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:
>>> FileSystemCounters
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:
>>> >> >>>>>>>> > HDFS_BYTES_READ=459
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:
>>> >> >>>>>>>> > FILE_BYTES_WRITTEN=189077
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:   File Input
>>> Format
>>> >> >>>>>>>> > Counters
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Bytes Read=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:   Map-Reduce
>>> Framework
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Map input
>>> >> >>>>>>>> > records=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Physical
>>> memory
>>> >> >>>>>>>> > (bytes)
>>> >> >>>>>>>> > snapshot=363053056
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Spilled
>>> Records=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     CPU time spent
>>> >> >>>>>>>> > (ms)=2110
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Total
>>> committed
>>> >> >>>>>>>> > heap
>>> >> >>>>>>>> > usage
>>> >> >>>>>>>> > (bytes)=553517056
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Virtual memory
>>> >> >>>>>>>> > (bytes)
>>> >> >>>>>>>> > snapshot=2344087552
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:     Map output
>>> >> >>>>>>>> > records=0
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapred.JobClient:
>>> >> >>>>>>>> > SPLIT_RAW_BYTES=404
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapreduce.ExportJobBase:
>>> Transferred 459
>>> >> >>>>>>>> > bytes in
>>> >> >>>>>>>> > 30.0642 seconds (15.2673 bytes/sec)
>>> >> >>>>>>>> > 14/05/13 10:27:13 INFO mapreduce.ExportJobBase: Exported 0
>>> >> >>>>>>>> > records.
>>> >> >>>>>>>> > 14/05/13 10:27:13 ERROR tool.ExportTool: Error during
>>> export:
>>> >> >>>>>>>> > Export job
>>> >> >>>>>>>> > failed!
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> >
>>> >> >>>>>>>> > On Mon, May 12, 2014 at 10:44 PM, Jarek Jarcec Cecho
>>> >> >>>>>>>> > <jarcec@apache.org>wrote:
>>> >> >>>>>>>> >
>>> >> >>>>>>>> > > The map task log contain entire executed query and lot of
>>> >> >>>>>>>> > > additional
>>> >> >>>>>>>> > > information and hence it's supper useful in such cases.
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > > Jarcec
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > > On Mon, May 12, 2014 at 02:59:56PM -0300, Leonardo
>>> Brambilla
>>> >> >>>>>>>> > > wrote:
>>> >> >>>>>>>> > > > Hi Jarek,
>>> >> >>>>>>>> > > >
>>> >> >>>>>>>> > > > thanks for replying, I don't have the logs. I'll see
>>> if I
>>> >> >>>>>>>> > > > can
>>> >> >>>>>>>> > > > run the
>>> >> >>>>>>>> > > task
>>> >> >>>>>>>> > > > again and then keep the logs.
>>> >> >>>>>>>> > > >
>>> >> >>>>>>>> > > > Anyway, I don't remember seeing anything else than this
>>> >> >>>>>>>> > > > SQLException
>>> >> >>>>>>>> > > about
>>> >> >>>>>>>> > > > missing parameter.
>>> >> >>>>>>>> > > >
>>> >> >>>>>>>> > > > Leo
>>> >> >>>>>>>> > > >
>>> >> >>>>>>>> > > >
>>> >> >>>>>>>> > > > On Sun, May 11, 2014 at 10:59 AM, Jarek Jarcec Cecho
>>> >> >>>>>>>> > > > <jarcec@apache.org
>>> >> >>>>>>>> > > >wrote:
>>> >> >>>>>>>> > > >
>>> >> >>>>>>>> > > > > Hi Leonardo,
>>> >> >>>>>>>> > > > > would you mind sharing with us task log from the
>>> failed
>>> >> >>>>>>>> > > > > map
>>> >> >>>>>>>> > > > > task?
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > > > > Jarcec
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > > > > On Sun, May 11, 2014 at 10:33:11AM -0300, Leonardo
>>> >> >>>>>>>> > > > > Brambilla
>>> >> >>>>>>>> > > > > wrote:
>>> >> >>>>>>>> > > > > > Hello, I am struggling to make it work, what is a
>>> >> >>>>>>>> > > > > > really
>>> >> >>>>>>>> > > > > > required
>>> >> >>>>>>>> > > > > feature.
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > I have a process that daily generates new data,
>>> this
>>> >> >>>>>>>> > > > > > data
>>> >> >>>>>>>> > > > > > needs to be
>>> >> >>>>>>>> > > > > > pushed to a table in Oracle, the table might
>>> already
>>> >> >>>>>>>> > > > > > have
>>> >> >>>>>>>> > > > > > same data
>>> >> >>>>>>>> > > from
>>> >> >>>>>>>> > > > > > previous loads. I need to avoid duplicating data
>>> on it.
>>> >> >>>>>>>> > > > > > Pretty common
>>> >> >>>>>>>> > > > > > scenario right? =)
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > I am using sqoop export for this, no special
>>> arguments,
>>> >> >>>>>>>> > > > > > just columns,
>>> >> >>>>>>>> > > > > > fields-terminated-by, table and db connection,
>>> plus the
>>> >> >>>>>>>> > > > > > argument
>>> >> >>>>>>>> > > > > > "update-mode allowinsert".
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > Now, when I also include the argument "update-key"
>>> with
>>> >> >>>>>>>> > > > > > a
>>> >> >>>>>>>> > > > > > comma
>>> >> >>>>>>>> > > separated
>>> >> >>>>>>>> > > > > > list of fields (which is the same for arg columns)
>>> I
>>> >> >>>>>>>> > > > > > get
>>> >> >>>>>>>> > > > > > the
>>> >> >>>>>>>> > > following
>>> >> >>>>>>>> > > > > > oracle driver error:
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > 14/05/07 16:00:03 INFO mapred.JobClient: Task Id :
>>> >> >>>>>>>> > > > > > attempt_201404190827_0928_m_000003_2, Status :
>>> FAILED
>>> >> >>>>>>>> > > > > > java.io.IOException: Can't export data, please
>>> check
>>> >> >>>>>>>> > > > > > task
>>> >> >>>>>>>> > > > > > tracker
>>> >> >>>>>>>> > > logs
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > > > >
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > > java.security.AccessController.doPrivileged(Native
>>> >> >>>>>>>> > > > > > Method)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > > javax.security.auth.Subject.doAs(Subject.java:415)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > > org.apache.hadoop.mapred.Child.main(Child.java:249)
>>> >> >>>>>>>> > > > > > Caused by: java.io.IOException:
>>> java.sql.SQLException:
>>> >> >>>>>>>> > > > > > Missing IN or
>>> >> >>>>>>>> > > OUT
>>> >> >>>>>>>> > > > > > parameter at index:: 4
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:220)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:46)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:639)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:84)
>>> >> >>>>>>>> > > > > >         ... 10 more
>>> >> >>>>>>>> > > > > > *Caused by: java.sql.SQLException: Missing IN or
>>> OUT
>>> >> >>>>>>>> > > > > > parameter at
>>> >> >>>>>>>> > > > > index:: 4*
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> oracle.jdbc.driver.OraclePreparedStatement.processCompletedBindRow(OraclePreparedStatement.java:1844)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> oracle.jdbc.driver.OraclePreparedStatement.addBatch(OraclePreparedStatement.java:10213)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> oracle.jdbc.driver.OraclePreparedStatementWrapper.addBatch(OraclePreparedStatementWrapper.java:1362)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.UpdateOutputFormat$UpdateRecordWriter.getPreparedStatement(UpdateOutputFormat.java:174)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.execUpdate(AsyncSqlRecordWriter.java:149)
>>> >> >>>>>>>> > > > > >         at
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>> > >
>>> org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWriter.java:215)
>>> >> >>>>>>>> > > > > >         ... 14 more
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > I'm using Sqoop 1.4.3 with hadoop1, also tried
>>> 1.4.4
>>> >> >>>>>>>> > > > > > with
>>> >> >>>>>>>> > > > > > same
>>> >> >>>>>>>> > > result. I
>>> >> >>>>>>>> > > > > > have the standard Oracle JDBC driver 6 with Java 7.
>>> >> >>>>>>>> > > > > > I went through all the documentation, Sqoop user
>>> guide
>>> >> >>>>>>>> > > > > > says this is
>>> >> >>>>>>>> > > > > > supported for built-in connector which I
>>> understand I
>>> >> >>>>>>>> > > > > > am
>>> >> >>>>>>>> > > > > > using.
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > Here is the full command:
>>> >> >>>>>>>> > > > > > $sqoopExecutable export \
>>> >> >>>>>>>> > > > > > --outdir $outdir \
>>> >> >>>>>>>> > > > > > --connect $connectionString --table $table_client
>>> >> >>>>>>>> > > > > > --username $dbUser
>>> >> >>>>>>>> > > > > > --password $dbUserPasswd \
>>> >> >>>>>>>> > > > > > --columns CLIENT_ID,EXP_ID,BUCKET_ID --update-key
>>> >> >>>>>>>> > > > > > CLIENT_ID,EXP_ID,BUCKET_ID \
>>> >> >>>>>>>> > > > > > --fields-terminated-by '\t' --update-mode
>>> allowinsert \
>>> >> >>>>>>>> > > > > > --export-dir $dataSource_client > $sqoopLog 2>&1
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > Can someone please shed some light on this?
>>> >> >>>>>>>> > > > > > Thank you in advance.
>>> >> >>>>>>>> > > > > >
>>> >> >>>>>>>> > > > > > Leo
>>> >> >>>>>>>> > > > >
>>> >> >>>>>>>> > >
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>
>>> >> >>>>>
>>> >> >>>>
>>> >> >>>
>>> >> >>
>>> >> >
>>> >
>>> >
>>>
>>
>>
>

Mime
View raw message