sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peleg, Eyal" <eyal.pe...@intel.com>
Subject RE: failed export command
Date Sun, 09 Jun 2013 12:58:37 GMT
13/06/09 05:49:44 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure.
Consider using -P instead.
13/06/09 05:49:44 ERROR sqoop.ConnFactory: Error loading ManagerFactory information from file
/usr/lib/sqoop/conf/managers.d/connectors: java.io.IOException: Could not load jar /usr/lib/
into JVM. (Could not find class com.cloudera.sqoop.manager.NetezzaManagerFactory.)
        at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:92)
        at com.cloudera.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:36)
        at org.apache.sqoop.ConnFactory.addManagersFromFile(ConnFactory.java:159)
        at org.apache.sqoop.ConnFactory.loadManagersFromConfDir(ConnFactory.java:218)
        at org.apache.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:83)
        at org.apache.sqoop.ConnFactory.<init>(ConnFactory.java:60)
        at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:36)
        at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:86)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
        at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
Caused by: java.lang.ClassNotFoundException: com.cloudera.sqoop.manager.NetezzaManagerFactory
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:247)
        at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:88)
        ... 15 more

13/06/09 05:49:44 ERROR sqoop.ConnFactory: Could not load ManagerFactory com.cloudera.sqoop.manager.NetezzaManagerFactory
(not found)
13/06/09 05:49:44 INFO manager.SqlManager: Using default fetchSize of 1000
13/06/09 05:49:44 INFO tool.CodeGenTool: Beginning code generation
13/06/09 05:49:44 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testing
AS t WHERE 1=0
13/06/09 05:49:44 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop/libexec/..
13/06/09 05:49:44 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/libexec/../hadoop-core.jar
Note: /tmp/sqoop-eyapeleg/compile/a480b3a617b2f64c5964790087de8c94/testing.java uses or overrides
a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/06/09 05:49:45 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-eyapeleg/compile/a480b3a617b2f64c5964790087de8c94/testing.java
to /home/eyapeleg/./testing.java
org.apache.commons.io.FileExistsException: Destination '/home/eyapeleg/./testing.java' already
exists
        at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2378)
        at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:97)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
        at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
13/06/09 05:49:45 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-eyapeleg/compile/a480b3a617b2f64c5964790087de8c94/testing.jar
13/06/09 05:49:45 INFO mapreduce.ExportJobBase: Beginning export of testing
13/06/09 05:49:51 INFO input.FileInputFormat: Total input paths to process : 1
13/06/09 05:49:51 INFO input.FileInputFormat: Total input paths to process : 1
13/06/09 05:49:51 INFO mapred.JobClient: Running job: job_201305130941_2944
13/06/09 05:49:52 INFO mapred.JobClient:  map 0% reduce 0%
13/06/09 05:50:08 INFO mapred.JobClient: Task Id : attempt_201305130941_2944_m_000000_0, Status
: FAILED
java.io.IOException: Could not parse record: aa a
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
        at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
        at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
...
...
        at org.apache.hadoop.mapred.MapTask
13/06/09 05:50:16 INFO mapred.JobClient: Task Id : attempt_201305130941_2944_m_000000_1, Status
: FAILED
java.io.IOException: Could not parse record: aa a
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
        at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
        at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
...
...
        at org.apache.hadoop.mapred.MapTask
13/06/09 05:50:23 INFO mapred.JobClient: Task Id : attempt_201305130941_2944_m_000000_2, Status
: FAILED
java.io.IOException: Could not parse record: aa a
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
        at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
        at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
...
...
        at org.apache.hadoop.mapred.MapTask
13/06/09 05:50:34 INFO mapred.JobClient: Job complete: job_201305130941_2944
13/06/09 05:50:34 INFO mapred.JobClient: Counters: 8
13/06/09 05:50:34 INFO mapred.JobClient:   Job Counters
13/06/09 05:50:34 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=33660
13/06/09 05:50:34 INFO mapred.JobClient:     Total time spent by all reduces waiting after
reserving slots (ms)=0
13/06/09 05:50:34 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving
slots (ms)=0
13/06/09 05:50:34 INFO mapred.JobClient:     Rack-local map tasks=3
13/06/09 05:50:34 INFO mapred.JobClient:     Launched map tasks=4
13/06/09 05:50:34 INFO mapred.JobClient:     Data-local map tasks=1
13/06/09 05:50:34 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
13/06/09 05:50:34 INFO mapred.JobClient:     Failed map tasks=1
13/06/09 05:50:35 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 48.7006 seconds (0
bytes/sec)
13/06/09 05:50:35 INFO mapreduce.ExportJobBase: Exported 0 records.
13/06/09 05:50:35 ERROR tool.ExportTool: Error during export: Export job failed!

From: abhijeet gaikwad [mailto:abygaikwad17@gmail.com]
Sent: Sunday, June 09, 2013 15:44
To: user@sqoop.apache.org
Subject: Re: failed export command

Can you run this command with --verbose option in the command and post whole console log here?

Thanks,
Abhijeet
My export command is as follows:

--table testing --export-dir  /user/eyapeleg/test --input-fields-terminated-by '\t'  --input-escaped-by
'\t'  --lines-terminated-by '\n'


*        note! I used tab delimited format.

*        I omitted the source name

I'm able to export the following table:

a              a
b             b

but  fail to export the next one:

aa           a
bb           a

I get a parse exception:

java.io.IOException: Could not parse record: aa a
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:80)
        at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
        at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
        at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
        at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
        at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)


---------------------------------------------------------------------
Intel Electronics Ltd.

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Mime
View raw message