Hi Arvind,

On Nov 29, 2011, at 10:52am, Arvind Prabhakar wrote:

I personally have not encountered this problem before. The trace
however looks very similar to the issue described in [1]. What Sqoop
is doing in that step is to create a jar from the compiled files and
for some reason it fails when it finds a nested directory within the
base directory which results in this NPE.

Yes, it looks like that same issue.

There was as sub-directory inside of the code generation dir, with a name == a single non breaking space char.

So it wasn't obvious that it existed, when we were trying to figure out what was different about that one cluster.

One thing that made this a bit more confusing was that the first entry in the logs that looked wrong was:

11/11/15 14:55:13 WARN orm.CompilationManager: IOException closing jar stream: java.util.zip.ZipException: ZIP file must have at least one entry

Which led us down a different/wrong path, trying to figure out why the class file wasn't found (since it clearly existed). This warning gets generated from the finally {} clause in the jar() method.

Regards,

-- Ken



Other than trying a different VM, I would suggest wiping the code
generation directory out and trying the command over again.

[1] http://bugs.sun.com/view_bug.do?bug_id=6698652

Thanks,
Arvind

On Mon, Nov 28, 2011 at 9:08 PM, Ken Krugler
<kkrugler_lists@transpac.com> wrote:
I've run into an odd issue, where on one of the clusters at a client, we get
this error while running Sqoop:

11/11/15 14:55:12 INFO orm.CompilationManager: HADOOP_HOME is
/usr/lib/hadoop
11/11/15 14:55:12 INFO orm.CompilationManager: Found hadoop core jar at:
/usr/lib/hadoop/hadoop-0.20.2-cdh3u0-core.jar
Note: /tmp/cgc-sqoop/owner_table.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
11/11/15 14:55:13 INFO orm.CompilationManager: Writing jar file:
/tmp/cgc-sqoop/owner.table.jar
11/11/15 14:55:13 WARN orm.CompilationManager: IOException closing jar
stream: java.util.zip.ZipException: ZIP file must have at least one entry
11/11/15 14:55:13 ERROR sqoop.Sqoop: Got exception running Sqoop:
java.lang.NullPointerException
java.lang.NullPointerException
at java.util.Arrays$ArrayList.<init>(Arrays.java:3357)
at java.util.Arrays.asList(Arrays.java:3343)
at
com.cloudera.sqoop.util.FileListing.getFileListingNoSort(FileListing.java:76)
at
com.cloudera.sqoop.util.FileListing.getFileListingNoSort(FileListing.java:82)
at com.cloudera.sqoop.util.FileListing.getFileListing(FileListing.java:67)
at
com.cloudera.sqoop.orm.CompilationManager.addClassFilesFromDir(CompilationManager.java:267)
at
com.cloudera.sqoop.orm.CompilationManager.jar(CompilationManager.java:324)
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:84)
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:218)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:228)

This only happens on one of the 8+ clusters.

Looking in the generated code directory, there's a valid owner_table.java
file, a valid owner_table.class file, but a 0 length owner.table.jar file.

All permissions seem fine for the directory being used to hold the generated
files.

This is on CDH3u0, and a patched version of the pre-released code base of
Sqoop 1.3.

Has anybody ever run into this before?

Thanks,

-- Ken


--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr





--------------------------
Ken Krugler
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr