sqoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ken Krugler <kkrugler_li...@transpac.com>
Subject Re: Odd problem with Sqoop & generated jar file
Date Wed, 30 Nov 2011 01:14:20 GMT
Hi Arvind,

On Nov 29, 2011, at 10:52am, Arvind Prabhakar wrote:

> I personally have not encountered this problem before. The trace
> however looks very similar to the issue described in [1]. What Sqoop
> is doing in that step is to create a jar from the compiled files and
> for some reason it fails when it finds a nested directory within the
> base directory which results in this NPE.

Yes, it looks like that same issue.

There was as sub-directory inside of the code generation dir, with a name == a single non
breaking space char.

So it wasn't obvious that it existed, when we were trying to figure out what was different
about that one cluster.

One thing that made this a bit more confusing was that the first entry in the logs that looked
wrong was:

11/11/15 14:55:13 WARN orm.CompilationManager: IOException closing jar stream: java.util.zip.ZipException:
ZIP file must have at least one entry

Which led us down a different/wrong path, trying to figure out why the class file wasn't found
(since it clearly existed). This warning gets generated from the finally {} clause in the
jar() method.

Regards,

-- Ken


> 
> Other than trying a different VM, I would suggest wiping the code
> generation directory out and trying the command over again.
> 
> [1] http://bugs.sun.com/view_bug.do?bug_id=6698652
> 
> Thanks,
> Arvind
> 
> On Mon, Nov 28, 2011 at 9:08 PM, Ken Krugler
> <kkrugler_lists@transpac.com> wrote:
>> I've run into an odd issue, where on one of the clusters at a client, we get
>> this error while running Sqoop:
>> 
>> 11/11/15 14:55:12 INFO orm.CompilationManager: HADOOP_HOME is
>> /usr/lib/hadoop
>> 11/11/15 14:55:12 INFO orm.CompilationManager: Found hadoop core jar at:
>> /usr/lib/hadoop/hadoop-0.20.2-cdh3u0-core.jar
>> Note: /tmp/cgc-sqoop/owner_table.java uses or overrides a deprecated API.
>> Note: Recompile with -Xlint:deprecation for details.
>> 11/11/15 14:55:13 INFO orm.CompilationManager: Writing jar file:
>> /tmp/cgc-sqoop/owner.table.jar
>> 11/11/15 14:55:13 WARN orm.CompilationManager: IOException closing jar
>> stream: java.util.zip.ZipException: ZIP file must have at least one entry
>> 11/11/15 14:55:13 ERROR sqoop.Sqoop: Got exception running Sqoop:
>> java.lang.NullPointerException
>> java.lang.NullPointerException
>> at java.util.Arrays$ArrayList.<init>(Arrays.java:3357)
>> at java.util.Arrays.asList(Arrays.java:3343)
>> at
>> com.cloudera.sqoop.util.FileListing.getFileListingNoSort(FileListing.java:76)
>> at
>> com.cloudera.sqoop.util.FileListing.getFileListingNoSort(FileListing.java:82)
>> at com.cloudera.sqoop.util.FileListing.getFileListing(FileListing.java:67)
>> at
>> com.cloudera.sqoop.orm.CompilationManager.addClassFilesFromDir(CompilationManager.java:267)
>> at
>> com.cloudera.sqoop.orm.CompilationManager.jar(CompilationManager.java:324)
>> at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:84)
>> at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
>> at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
>> at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
>> at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:218)
>> at com.cloudera.sqoop.Sqoop.main(Sqoop.java:228)
>> 
>> This only happens on one of the 8+ clusters.
>> 
>> Looking in the generated code directory, there's a valid owner_table.java
>> file, a valid owner_table.class file, but a 0 length owner.table.jar file.
>> 
>> All permissions seem fine for the directory being used to hold the generated
>> files.
>> 
>> This is on CDH3u0, and a patched version of the pre-released code base of
>> Sqoop 1.3.
>> 
>> Has anybody ever run into this before?
>> 
>> Thanks,
>> 
>> -- Ken
>> 
>> 
>> --------------------------
>> Ken Krugler
>> http://www.scaleunlimited.com
>> custom big data solutions & training
>> Hadoop, Cascading, Mahout & Solr
>> 
>> 
>> 
>> 

--------------------------
Ken Krugler
http://www.scaleunlimited.com
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr





Mime
View raw message