mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: Classpath question
Date Sat, 11 Sep 2010 10:39:47 GMT
IIRC this packaging mechanism only works if you add "Class-Path" entries to
META-INF/MANIFEST.MF that specify the location of the .jar within the
archive.

I prefer to simply re-package all dependencies together, like Mahout does.

On Sat, Sep 11, 2010 at 2:44 AM, Mark <static.void.dev@gmail.com> wrote:

>  Perhaps this a better place to post this (Originally posted to Hadoop)
>
>
> If I submit a jar that has a lib directory that contains a bunch of jars,
> shouldn't those jars be in the classpath and available to all nodes?
>
> The reason I ask this is because I am trying to submit a jar myjar.jar that
> has the following structure
>
> --src
>  \.... (My source classes)
> -- lib
>  \
>   -- mahout-collections-0.3.jar
>   -- mahout-core-0.3.jar
>   -- mahout-math-0.3.jar
>   -- hbase-0.20.0.jar
>   -- commons-cli-2.0-mahout.jar
>
> Now the job I am trying to run is actually part of mahout-core-0.3... not
> src. The job starts but then fails with the following error
>
> 10/09/10 03:15:13 INFO mapred.JobClient: Task Id :
> attempt_201009100306_0003_r_000000_0, Status : FAILED
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
>    at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
>    at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:550)
>    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
>    at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.reflect.InvocationTargetException
>    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>    at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>    at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>    at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
>    at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
>    ... 3 more
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/mahout/math/map/OpenObjectIntHashMap
>    at
> org.apache.mahout.fpm.pfpgrowth.ParallelFPGrowthReducer.<init>(ParallelFPGrowthReducer.java:56)
>    ... 8 more
>
>
> Apparently it cant find the
> org/apache/mahout/math/map/OpenObjectIntHashMap.class although that class is
> definitely in mahout-collections-0.3.jar. Is this a problem because the
> mahout-core-0.3.jar doesn't have a lib directory?
>
> What is an easy way around this?
>
> Thanks
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message