mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gokul Pillai <gokoolt...@gmail.com>
Subject Re: No class definition found for org.apache.mahout.math.Vector when invoking K-means example
Date Fri, 09 Jul 2010 23:16:29 GMT
Peter:
I just noticed that there has been a new patch applied by Drew yesterday for
this. (https://issues.apache.org/jira/browse/MAHOUT-426)

The last comment on the thread says "Integrated in Mahout-Quality #128 (See
http://hudson.zones.apache.org/hudson/job/Mahout-Quality/128/)"

Does it mean that if I did a "mvn-U" on the trunk, I should be able to get
the fix and then not run into the issue ?

Regards
gokul

On Thu, Jul 1, 2010 at 2:31 PM, Gokul Pillai <gokooltech@gmail.com> wrote:

> The version on my instance is 0.20.1+152.
> I am guessing if the jar file name is "hadoop-0.20.1+152-fairscheduler.jar"
> then that is the version.
>
>
> On Thu, Jul 1, 2010 at 11:11 AM, Peter M. Goldstein <
> peter_m_goldstein@yahoo.com> wrote:
>
>> Gokul,
>>
>> What version of Hadoop are you using?  I'm running with 0.20+320, and I
>> don't run into this issue.
>>
>> Thanks.
>>
>> Regards,
>>
>> Peter
>>
>> -----Original Message-----
>> From: Gokul Pillai [mailto:gokooltech@gmail.com]
>> Sent: Thursday, July 01, 2010 10:29 AM
>> To: user@mahout.apache.org
>> Subject: Re: No class definition found for org.apache.mahout.math.Vector
>> when invoking K-means example
>>
>> Hello Peter:
>> When i run that command, I get the following although different error:
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/commons/cli2/OptionException
>>    at java.lang.Class.forName0(Native Method)
>>    at java.lang.Class.forName(Class.java:247)
>>    at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.commons.cli2.OptionException
>>    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>
>> Digging into this, I find that hadoop 0.20 ships with
>> "commons-cli-1.2.jar"
>> but mahout 0.40 Snapshot version depends on "commons-cli-2.0-mahout.jar"
>> that is there in the $MAHOUT_HOME/trunk/examples/target/ folder once you
>> "mvn install" in the  "examples" folder.
>>
>> So i replaced the jar file in hadoop/lib with this one and then it goes
>> back
>> to the ClassNotFound error for Vector class.
>>
>> Regards
>> Gokul
>>
>> On Thu, Jul 1, 2010 at 9:29 AM, Peter M. Goldstein <
>> peter_m_goldstein@yahoo.com> wrote:
>>
>> > Hi Gokul,
>> >
>> > Using the Mahout on Amazon EC2 directions I'm able to get a little
>> farther
>> > than this.  Specifically, using the job file created when building the
>> > examples I don't get any ClassNotFoundExceptions.  Instead I get an
>> > InvalidInputException (presumably because I didn't prepare any input
>> data
>> > for the job).  If you provide some appropriately formatted candidate
>> data
>> > I'd be happy to try and run a complete test.
>> >
>> > In my test I'm using the command line:
>> >
>> > $HADOOP_HOME/bin/hadoop jar
>> > $MAHOUT_HOME/examples/target/mahout-examples-0.4-SNAPSHOT.job
>> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
>> >
>> > Again, it's critical to build and use the examples Job file.  If I
>> attempt
>> > to use the core Job file instead key classes appear to be absent.
>> >
>> > Hope that helps.
>> >
>> > --Peter
>> >
>> > -----Original Message-----
>> > From: Gokul Pillai [mailto:gokooltech@gmail.com]
>> > Sent: Wednesday, June 30, 2010 2:35 PM
>> > To: user@mahout.apache.org
>> > Subject: Re: No class definition found for org.apache.mahout.math.Vector
>> > when invoking K-means example
>> >
>> > Peter
>> > Thank you for your response. However, after making your changes, it
>> still
>> > does not work for me.
>> > In fact I tried another way which is:
>> > 1. create a lib folder within my Job Jar-file and put the
>> > "org.apache.mahout.math.Vector" and other dependent jars.
>> > 2. Launch the command via "hadoop jar ./mahout-examples-with-libs.jar
>> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job"
>> >
>> > However, I still get the error:
>> > "Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
>> >    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)"
>> >
>> >
>> > Another thing I tried is modify the $HADOOP_HOME/conf/hadoop-env.sh and
>> > edit
>> > the HADOOP_CLASSPATH to include the dependent libraries. Even that does
>> not
>> > work.
>> >
>> > Regards
>> > Gokul
>> >
>> > On Mon, Jun 28, 2010 at 3:43 PM, Peter M. Goldstein <
>> > peter_m_goldstein@yahoo.com> wrote:
>> >
>> > > Hi Gokul,
>> > >
>> > > You may want to see the following bug reports:
>> > >
>> > > https://issues.apache.org/jira/browse/MAHOUT-426
>> > >
>> > > https://issues.apache.org/jira/browse/MAHOUT-427
>> > >
>> > > https://issues.apache.org/jira/browse/MAHOUT-428
>> > >
>> > > I ran into similar issues as you and had to some minor patching to get
>> > > everything to work.
>> > >
>> > > Also, make sure you run an explicit "mvn install" from within the
>> > examples
>> > > directory.  I had to do this to get the JOB files to build correctly.
>> > > Getting the JOB files to build correctly solves (sort of) issues 426
>> and
>> > > 427, so you only need to apply the patch described in 428.
>> > >
>> > > Hope that helps.
>> > >
>> > > Regards,
>> > >
>> > > Peter
>> > >
>> > > -----Original Message-----
>> > > From: Gokul Pillai [mailto:gokooltech@gmail.com]
>> > > Sent: Monday, June 28, 2010 3:27 PM
>> > > To: user@mahout.apache.org
>> > > Subject: No class definition found for org.apache.mahout.math.Vector
>> when
>> > > invoking K-means example
>> > >
>> > > I made a new installation of Hadoop 0.20 from Cloudera.
>> > > I then installed Mahout 0.4-SNAPSHOT by doing a SVN checkout this
>> morning
>> > > and ran mvn install on core, utils and examples.
>> > > I tried running the KMeans Clustering example by doing the following
>> as
>> > > mentioned in the tutorial:
>> > > hadoop jar
>> > ~/mahout/trunk/examples/target/mahout-examples-0.4-SNAPSHOT.jar
>> > > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
>> > >
>> > > And I keep getting this error although I have explicitly set my
>> Classpath
>> > > to
>> > >
>> > >
>> >
>> >
>>
>> "/usr/lib/hadoop/lib/mahout-collections-1.0.jar:/usr/lib/hadoop/lib/mahout-m
>> > >
>> > >
>> >
>> >
>>
>> ath-0.4-SNAPSHOT.jar:/usr/lib/hadoop/lib/mahout-utils-0.4-SNAPSHOT.jar:/usr/
>> > > lib/hadoop/lib/mahout-core-0.4-SNAPSHOT.jar":
>> > >
>> > > 10/06/28 15:19:21 INFO mapred.JobClient: Task Id :
>> > > attempt_201006280959_0027_r_000000_1, Status : FAILED
>> > > java.lang.RuntimeException: java.lang.ClassNotFoundException:
>> > > org.apache.mahout.clustering.kmeans.KMeansCombiner
>> > >    at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:868)
>> > >    at
>> > >
>> > >
>> >
>> >
>>
>> org.apache.hadoop.mapreduce.JobContext.getCombinerClass(JobContext.java:169)
>> > >    at
>> org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1107)
>> > >    at
>> > >
>> > >
>> >
>> >
>>
>> org.apache.hadoop.mapred.ReduceTask$ReduceCopier.<init>(ReduceTask.java:1780
>> > > )
>> > >    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:375)
>> > >    at org.apache.hadoop.mapred.Child.main(Child.java:170)
>> > > Caused by: java.lang.ClassNotFoundException:
>> > > org.apache.mahout.clustering.kmeans.KMeansCombiner
>> > >    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> > >    at java.security.AccessController.doPrivileged(Native Method)
>> > >    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> > >    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> > >    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> > >    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
>> > >    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
>> > >    at java.lang.Class.forName0(Native Method)
>> > >    at java.lang.Class.forName(Class.java:247)
>> > >    at
>> > >
>> >
>>
>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:815)
>> > >    at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)
>> > >    ... 5 more
>> > >
>> > >
>> > > Any help would be appreciated.
>> > >
>> > > Regards
>> > > Gokul
>> > >
>> > >
>> > >
>> >
>> >
>> >
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message