sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jarek Jarcec Cecho <jar...@apache.org>
Subject Re: sqoop export - parse exception
Date Sun, 09 Jun 2013 16:01:54 GMT
Hi Peleg,
this seems to me as a data corruption issue, would you mind sharing with us the table schema
and example small input file that trigger the issue? I'm expecting that number of columns
in the table and in the file will be different.

What Sqoop version are you using? We've done significant improvements in Sqoop 1.4.3 (for
example SQOOP-720) that are simplifying triaging of such issues, so you might consider upgrade.

Jarcec

On Sun, Jun 09, 2013 at 03:52:46PM +0000, Peleg, Eyal wrote:
> First, thx for the quick response..
> 
> 
> The command:
> sqoop export ... --table testing --export-dir  /user/eyapeleg/test --input-fields-terminated-by
'\t' --lines-terminated-by '\n' --verbose ...
> 
> 
> the exception:
> java.util.NoSuchElementException
> 
> 
> the log:
> 
> 13/06/09 08:48:19 WARN tool.BaseSqoopTool: Setting your password on the command-line
is insecure. Consider using -P instead.
> 13/06/09 08:48:19 ERROR sqoop.ConnFactory: Error loading ManagerFactory information from
file /usr/lib/sqoop/conf/managers.d/connectors: java.io.IOException: Could not load jar /usr/lib/
into JVM. (Could not find class com.cloudera.sqoop.manager.NetezzaManagerFactory.)
>         at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:92)
>         at com.cloudera.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:36)
>         at org.apache.sqoop.ConnFactory.addManagersFromFile(ConnFactory.java:159)
>         at org.apache.sqoop.ConnFactory.loadManagersFromConfDir(ConnFactory.java:218)
>         at org.apache.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:83)
>         at org.apache.sqoop.ConnFactory.<init>(ConnFactory.java:60)
>         at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:36)
>         at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:202)
>         at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:86)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> Caused by: java.lang.ClassNotFoundException: com.cloudera.sqoop.manager.NetezzaManagerFactory
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>         at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:627)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>         at java.lang.Class.forName0(Native Method)
>         at java.lang.Class.forName(Class.java:247)
>         at org.apache.sqoop.util.ClassLoaderStack.addJarFile(ClassLoaderStack.java:88)
>         ... 15 more
> 
> 13/06/09 08:48:19 ERROR sqoop.ConnFactory: Could not load ManagerFactory com.cloudera.sqoop.manager.NetezzaManagerFactory
(not found)
> 13/06/09 08:48:19 INFO manager.SqlManager: Using default fetchSize of 1000
> 13/06/09 08:48:19 INFO tool.CodeGenTool: Beginning code generation
> 13/06/09 08:48:19 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM testing
AS t WHERE 1=0
> 13/06/09 08:48:19 INFO orm.CompilationManager: HADOOP_HOME is /usr/lib/hadoop/libexec/..
> 13/06/09 08:48:19 INFO orm.CompilationManager: Found hadoop core jar at: /usr/lib/hadoop/libexec/../hadoop-core.jar
> Note: /tmp/sqoop-eyapeleg/compile/637c112af558b7f914cfbd4024cbdb82/testing.java uses
or overrides a deprecated API.
> Note: Recompile with -Xlint:deprecation for details.
> 13/06/09 08:48:21 ERROR orm.CompilationManager: Could not rename /tmp/sqoop-eyapeleg/compile/637c112af558b7f914cfbd4024cbdb82/testing.java
to /home/eyapeleg/./testing.java
> org.apache.commons.io.FileExistsException: Destination '/home/eyapeleg/./testing.java'
already exists
>         at org.apache.commons.io.FileUtils.moveFile(FileUtils.java:2378)
>         at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:227)
>         at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
>         at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:64)
>         at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:97)
>         at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
>         at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
>         at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
>         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
> 13/06/09 08:48:21 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-eyapeleg/compile/637c112af558b7f914cfbd4024cbdb82/testing.jar
> 13/06/09 08:48:21 INFO mapreduce.ExportJobBase: Beginning export of testing
> 13/06/09 08:48:26 INFO input.FileInputFormat: Total input paths to process : 1
> 13/06/09 08:48:26 INFO input.FileInputFormat: Total input paths to process : 1
> 13/06/09 08:48:26 INFO mapred.JobClient: Running job: job_201305130941_2945
> 13/06/09 08:48:27 INFO mapred.JobClient:  map 0% reduce 0%
> 13/06/09 08:48:46 INFO mapred.JobClient: Task Id : attempt_201305130941_2945_m_000000_0,
Status : FAILED
> java.util.NoSuchElementException
>         at java.util.AbstractList$Itr.next(AbstractList.java:350)
>         at testing.__loadFromFields(testing.java:194)
>         at testing.parse(testing.java:143)
>         at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:77)
>         at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>         at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
>         at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
>         at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> ...
> 13/06/09 08:48:52 INFO mapred.JobClient: Task Id : attempt_201305130941_2945_m_000000_1,
Status : FAILED
> java.util.NoSuchElementException
>         at java.util.AbstractList$Itr.next(AbstractList.java:350)
>         at testing.__loadFromFields(testing.java:194)
>         at testing.parse(testing.java:143)
>         at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:77)
>         at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>         at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
>         at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
>         at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> ...
> 13/06/09 08:49:02 INFO mapred.JobClient: Task Id : attempt_201305130941_2945_m_000000_2,
Status : FAILED
> java.util.NoSuchElementException
>         at java.util.AbstractList$Itr.next(AbstractList.java:350)
>         at testing.__loadFromFields(testing.java:194)
>         at testing.parse(testing.java:143)
>         at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:77)
>         at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
>         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>         at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:183)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper_aroundBody4(MapTask.java:813)
>         at org.apache.hadoop.mapred.MapTask$AjcClosure5.run(MapTask.java:1)
>         at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> ...
> 13/06/09 08:49:17 INFO mapred.JobClient: Job complete: job_201305130941_2945
> 13/06/09 08:49:17 INFO mapred.JobClient: Counters: 8
> 13/06/09 08:49:17 INFO mapred.JobClient:   Job Counters
> 13/06/09 08:49:17 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=34661
> 13/06/09 08:49:17 INFO mapred.JobClient:     Total time spent by all reduces waiting
after reserving slots (ms)=0
> 13/06/09 08:49:17 INFO mapred.JobClient:     Total time spent by all maps waiting after
reserving slots (ms)=0
> 13/06/09 08:49:17 INFO mapred.JobClient:     Rack-local map tasks=3
> 13/06/09 08:49:17 INFO mapred.JobClient:     Launched map tasks=4
> 13/06/09 08:49:17 INFO mapred.JobClient:     Data-local map tasks=1
> 13/06/09 08:49:17 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
> 13/06/09 08:49:17 INFO mapred.JobClient:     Failed map tasks=1
> 13/06/09 08:49:17 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 55.4027 seconds
(0 bytes/sec)
> 13/06/09 08:49:17 INFO mapreduce.ExportJobBase: Exported 0 records.
> 13/06/09 08:49:17 ERROR tool.ExportTool: Error during export: Export job failed! 
> ---------------------------------------------------------------------
> Intel Electronics Ltd.
> 
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
> 

Mime
View raw message