mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gokul Pillai <gokoolt...@gmail.com>
Subject Re: No class definition found for org.apache.mahout.math.Vector when invoking K-means example
Date Fri, 09 Jul 2010 23:34:39 GMT
To clarify, do the fixes relate to only Java code or is there any script in
the mahout/trunk/bin folder too?
If so, how does one get that from the repo?

On Fri, Jul 9, 2010 at 4:16 PM, Gokul Pillai <gokooltech@gmail.com> wrote:

> Peter:
> I just noticed that there has been a new patch applied by Drew yesterday
> for this. (https://issues.apache.org/jira/browse/MAHOUT-426)
>
> The last comment on the thread says "Integrated in Mahout-Quality #128
> (See http://hudson.zones.apache.org/hudson/job/Mahout-Quality/128/)"
>
> Does it mean that if I did a "mvn-U" on the trunk, I should be able to get
> the fix and then not run into the issue ?
>
> Regards
> gokul
>
> On Thu, Jul 1, 2010 at 2:31 PM, Gokul Pillai <gokooltech@gmail.com> wrote:
>
>> The version on my instance is 0.20.1+152.
>> I am guessing if the jar file name is
>> "hadoop-0.20.1+152-fairscheduler.jar" then that is the version.
>>
>>
>> On Thu, Jul 1, 2010 at 11:11 AM, Peter M. Goldstein <
>> peter_m_goldstein@yahoo.com> wrote:
>>
>>> Gokul,
>>>
>>> What version of Hadoop are you using?  I'm running with 0.20+320, and I
>>> don't run into this issue.
>>>
>>> Thanks.
>>>
>>> Regards,
>>>
>>> Peter
>>>
>>> -----Original Message-----
>>> From: Gokul Pillai [mailto:gokooltech@gmail.com]
>>> Sent: Thursday, July 01, 2010 10:29 AM
>>> To: user@mahout.apache.org
>>> Subject: Re: No class definition found for org.apache.mahout.math.Vector
>>> when invoking K-means example
>>>
>>> Hello Peter:
>>> When i run that command, I get the following although different error:
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/apache/commons/cli2/OptionException
>>>    at java.lang.Class.forName0(Native Method)
>>>    at java.lang.Class.forName(Class.java:247)
>>>    at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.commons.cli2.OptionException
>>>    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>>
>>> Digging into this, I find that hadoop 0.20 ships with
>>> "commons-cli-1.2.jar"
>>> but mahout 0.40 Snapshot version depends on "commons-cli-2.0-mahout.jar"
>>> that is there in the $MAHOUT_HOME/trunk/examples/target/ folder once you
>>> "mvn install" in the  "examples" folder.
>>>
>>> So i replaced the jar file in hadoop/lib with this one and then it goes
>>> back
>>> to the ClassNotFound error for Vector class.
>>>
>>> Regards
>>> Gokul
>>>
>>> On Thu, Jul 1, 2010 at 9:29 AM, Peter M. Goldstein <
>>> peter_m_goldstein@yahoo.com> wrote:
>>>
>>> > Hi Gokul,
>>> >
>>> > Using the Mahout on Amazon EC2 directions I'm able to get a little
>>> farther
>>> > than this.  Specifically, using the job file created when building the
>>> > examples I don't get any ClassNotFoundExceptions.  Instead I get an
>>> > InvalidInputException (presumably because I didn't prepare any input
>>> data
>>> > for the job).  If you provide some appropriately formatted candidate
>>> data
>>> > I'd be happy to try and run a complete test.
>>> >
>>> > In my test I'm using the command line:
>>> >
>>> > $HADOOP_HOME/bin/hadoop jar
>>> > $MAHOUT_HOME/examples/target/mahout-examples-0.4-SNAPSHOT.job
>>> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
>>> >
>>> > Again, it's critical to build and use the examples Job file.  If I
>>> attempt
>>> > to use the core Job file instead key classes appear to be absent.
>>> >
>>> > Hope that helps.
>>> >
>>> > --Peter
>>> >
>>> > -----Original Message-----
>>> > From: Gokul Pillai [mailto:gokooltech@gmail.com]
>>> > Sent: Wednesday, June 30, 2010 2:35 PM
>>> > To: user@mahout.apache.org
>>> > Subject: Re: No class definition found for
>>> org.apache.mahout.math.Vector
>>> > when invoking K-means example
>>> >
>>> > Peter
>>> > Thank you for your response. However, after making your changes, it
>>> still
>>> > does not work for me.
>>> > In fact I tried another way which is:
>>> > 1. create a lib folder within my Job Jar-file and put the
>>> > "org.apache.mahout.math.Vector" and other dependent jars.
>>> > 2. Launch the command via "hadoop jar ./mahout-examples-with-libs.jar
>>> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job"
>>> >
>>> > However, I still get the error:
>>> > "Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector
>>> >    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)"
>>> >
>>> >
>>> > Another thing I tried is modify the $HADOOP_HOME/conf/hadoop-env.sh and
>>> > edit
>>> > the HADOOP_CLASSPATH to include the dependent libraries. Even that does
>>> not
>>> > work.
>>> >
>>> > Regards
>>> > Gokul
>>> >
>>> > On Mon, Jun 28, 2010 at 3:43 PM, Peter M. Goldstein <
>>> > peter_m_goldstein@yahoo.com> wrote:
>>> >
>>> > > Hi Gokul,
>>> > >
>>> > > You may want to see the following bug reports:
>>> > >
>>> > > https://issues.apache.org/jira/browse/MAHOUT-426
>>> > >
>>> > > https://issues.apache.org/jira/browse/MAHOUT-427
>>> > >
>>> > > https://issues.apache.org/jira/browse/MAHOUT-428
>>> > >
>>> > > I ran into similar issues as you and had to some minor patching to
>>> get
>>> > > everything to work.
>>> > >
>>> > > Also, make sure you run an explicit "mvn install" from within the
>>> > examples
>>> > > directory.  I had to do this to get the JOB files to build correctly.
>>> > > Getting the JOB files to build correctly solves (sort of) issues 426
>>> and
>>> > > 427, so you only need to apply the patch described in 428.
>>> > >
>>> > > Hope that helps.
>>> > >
>>> > > Regards,
>>> > >
>>> > > Peter
>>> > >
>>> > > -----Original Message-----
>>> > > From: Gokul Pillai [mailto:gokooltech@gmail.com]
>>> > > Sent: Monday, June 28, 2010 3:27 PM
>>> > > To: user@mahout.apache.org
>>> > > Subject: No class definition found for org.apache.mahout.math.Vector
>>> when
>>> > > invoking K-means example
>>> > >
>>> > > I made a new installation of Hadoop 0.20 from Cloudera.
>>> > > I then installed Mahout 0.4-SNAPSHOT by doing a SVN checkout this
>>> morning
>>> > > and ran mvn install on core, utils and examples.
>>> > > I tried running the KMeans Clustering example by doing the following
>>> as
>>> > > mentioned in the tutorial:
>>> > > hadoop jar
>>> > ~/mahout/trunk/examples/target/mahout-examples-0.4-SNAPSHOT.jar
>>> > > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
>>> > >
>>> > > And I keep getting this error although I have explicitly set my
>>> Classpath
>>> > > to
>>> > >
>>> > >
>>> >
>>> >
>>>
>>> "/usr/lib/hadoop/lib/mahout-collections-1.0.jar:/usr/lib/hadoop/lib/mahout-m
>>> > >
>>> > >
>>> >
>>> >
>>>
>>> ath-0.4-SNAPSHOT.jar:/usr/lib/hadoop/lib/mahout-utils-0.4-SNAPSHOT.jar:/usr/
>>> > > lib/hadoop/lib/mahout-core-0.4-SNAPSHOT.jar":
>>> > >
>>> > > 10/06/28 15:19:21 INFO mapred.JobClient: Task Id :
>>> > > attempt_201006280959_0027_r_000000_1, Status : FAILED
>>> > > java.lang.RuntimeException: java.lang.ClassNotFoundException:
>>> > > org.apache.mahout.clustering.kmeans.KMeansCombiner
>>> > >    at
>>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:868)
>>> > >    at
>>> > >
>>> > >
>>> >
>>> >
>>>
>>> org.apache.hadoop.mapreduce.JobContext.getCombinerClass(JobContext.java:169)
>>> > >    at
>>> org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1107)
>>> > >    at
>>> > >
>>> > >
>>> >
>>> >
>>>
>>> org.apache.hadoop.mapred.ReduceTask$ReduceCopier.<init>(ReduceTask.java:1780
>>> > > )
>>> > >    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:375)
>>> > >    at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>> > > Caused by: java.lang.ClassNotFoundException:
>>> > > org.apache.mahout.clustering.kmeans.KMeansCombiner
>>> > >    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>> > >    at java.security.AccessController.doPrivileged(Native Method)
>>> > >    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>> > >    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>> > >    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> > >    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
>>> > >    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
>>> > >    at java.lang.Class.forName0(Native Method)
>>> > >    at java.lang.Class.forName(Class.java:247)
>>> > >    at
>>> > >
>>> >
>>>
>>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:815)
>>> > >    at
>>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866)
>>> > >    ... 5 more
>>> > >
>>> > >
>>> > > Any help would be appreciated.
>>> > >
>>> > > Regards
>>> > > Gokul
>>> > >
>>> > >
>>> > >
>>> >
>>> >
>>> >
>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message