mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Lyubimov <dlie...@gmail.com>
Subject Re: Running KMeans Directly from Java Program on Hadoop-0.20.2 and 'Vector' ClassNotFound error
Date Wed, 12 Jan 2011 05:06:55 GMT
I guess this stands close to my question in another email thread.

There is a MR option for hadoop to grab extra jars but i never used it in
conjunction with ToolRunner which is what Mahout is presumably uses
everywhere. Besides, I am not sure that i am not going to be overriding
results of JobConf.setJar() which i think most Mahout jobs use to set the MR
classpath.

i guess a little experimenation is a key, but i am fairly sure placing jars
into $HADOOP_HOME/lib is not going to help by default.

On Tue, Jan 11, 2011 at 9:01 PM, Lokendra Singh <lsingh.969@gmail.com>wrote:

> Hi,
>
> @Sean: I am actually directly trying to run KMeans from
> KMeansDriver.runJob() method in my java program (and not using hadoop
> classes like RunJar etc), hence even putting .job file in my classpath is
> not making any difference.
>
> @Ted: If the classpath jars are not visible to hadoop internally,  what is
> the global Classpath used by hadoop and how to change it ? I had tried
> putting the required jars in '$HADOOP_HOME/lib' also, thinking that this
> would be a central repository of libs, but no use.
>
>
> Regards
> Lokendra
>
>
> On Tue, Jan 11, 2011 at 10:01 PM, Sean Owen <srowen@gmail.com> wrote:
>
> > Yes, you should be using the "job" .jar file, not the regular .jar
> > file. The "job" file is a .jar file which contains all dependencies.
> > Are you doing this?
> >
> > On Tue, Jan 11, 2011 at 4:29 PM, Ted Dunning <ted.dunning@gmail.com>
> > wrote:
> > >
> > > I don't know the ultimate cause, but the proximal cause is probably
> that
> > the
> > > mahout-math.jar is not
> > > being passed to the mappers and reducers.  The fact that it is in your
> > class
> > > path is only part of the
> > > battle because hadoop has to know that it needs to pass this jar to the
> > > other machines running your
> > > program.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message