mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Mundi <cmu...@gmail.com>
Subject Re: Is anyone running Mahout 0.8 against Hadoop 2.0.5-alpha
Date Wed, 24 Jul 2013 18:14:01 GMT
Thank you, Suneel.  I will go back to hadoop 1.x for now.

I hope mahout will add support for hadoop 2 soon, as I have a use case for
federation.

Cheers!
On Jul 24, 2013 12:41 AM, "Suneel Marthi" <suneel_marthi@yahoo.com> wrote:

> Mahout 0.8 is compatible with Hadoop 1.x (and not Hadoop 2.x). Hence the
> issue s u r seeing.
>
> Sent from my iPhone
>
> On Jul 24, 2013, at 2:44 AM, Carlos Mundi <cmundi@gmail.com> wrote:
>
> > Is anyone running Mahout 0.8 against Hadoop 2.0.5-alpha?
> >
> > I think I have a hadoop configuration issue.  I've seen similar
> discussions
> > but have not found (or recognized) a solution yet.  Rather than tell you
> > everything I've tried, here's a simple example:
> >
> > First, I grab the 0.8 tag and build it:
> >
> > cd /home/hduser/Mahout
> > svn co http://svn.apache.org/repos/asf/mahout/tags/mahout-0.8/
> > export $MAHOUT_HOME=cd /home/hduser/Mahout/mahout-0.8
> > mvn -DskipTests clean install
> >
> > The maven build succeeds.
> > I start a single-cluster instance of hadoop and verify it is up.
> >
> > Now I attempt a canopy clustering, using the helpfully provided script:
> >
> > hduser@ubuntu:~/Mahout/mahout-0.8/examples/bin$
> > ./cluster-syntheticcontrol.sh
> > Please select a number to choose the corresponding clustering algorithm
> > 1. canopy clustering
> > 2. kmeans clustering
> > 3. fuzzykmeans clustering
> > 4. dirichlet clustering
> > 5. meanshift clustering
> > Enter your choice : 1
> > ok. You chose 1 and we'll use canopy Clustering
> > creating work directory at /tmp/mahout-work-hduser
> > Downloading Synthetic control data
> >  % Total    % Received % Xferd  Average Speed   Time    Time     Time
> > Current
> >                                 Dload  Upload   Total   Spent    Left
> > Speed
> > 100  281k  100  281k    0     0   598k      0 --:--:-- --:--:-- --:--:--
> > 707k
> > Checking the health of DFS...
> > 13/07/23 23:01:24 WARN util.NativeCodeLoader: Unable to load
> native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > Found 2 items
> > drwxr-xr-x   - hduser supergroup          0 2013-07-20 08:24 guten
> > drwxr-xr-x   - hduser supergroup          0 2013-07-23 22:54 testdata
> > DFS is healthy...
> > Uploading Synthetic control data to HDFS
> > rmr: DEPRECATED: Please use 'rm -r' instead.
> > 13/07/23 23:01:26 WARN util.NativeCodeLoader: Unable to load
> native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > Deleted testdata
> > 13/07/23 23:01:28 WARN util.NativeCodeLoader: Unable to load
> native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > 13/07/23 23:01:30 WARN util.NativeCodeLoader: Unable to load
> native-hadoop
> > library for your platform... using builtin-java classes where applicable
> > Successfully Uploaded Synthetic control data to HDFS
> > MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
> > Running on hadoop, using /usr/local/hadoop/bin/hadoop and
> > HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop
> > MAHOUT-JOB:
> >
> /home/hduser/Mahout/mahout-0.8/examples/target/mahout-examples-0.8-job.jar
> > 13/07/23 23:01:51 WARN driver.MahoutDriver: No
> > org.apache.mahout.clustering.syntheticcontrol.canopy.Job.props found on
> > classpath, will use command-line arguments only
> > Exception in thread "main" java.lang.NoSuchMethodError:
> > org.apache.hadoop.util.ProgramDriver.driver([Ljava/lang/String;)V
> >    at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:194)
> >    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >    at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >    at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >    at java.lang.reflect.Method.invoke(Method.java:606)
> >    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
> >
> >
> > Hmmm... looking at the pom.xml, I see that Mahout 0.8 calls for hadoop
> > 1.1.2.  I also see that there is no POM for hadoop 2.x.y at
> > http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-core/*
> >
> > Consulting both of
> >
> > http://hadoop.apache.org/docs/r1.1.2/api/index.html
> > http://hadoop.apache.org/docs/r2.0.5-alpha/api/index.html
> >
> > shows that org.apache.hadoop.util has been seriously reworked for 2.x
> >
> > Perhaps it is as simple as rewriting the examples to use the new hadoop
> > API?  I've already been working around some trivial API name changes in
> > hadoop 2.0, so maybe ProgramDriver just got refactored?
> >
> > I need the federation in hadoop 2.x so I am hoping someone has been down
> > this path!
> >
> > So... Is anyone running Mahout 0.8 against Hadoop 2.x.y ?
> >
> > I am learning the ecosystem, and I appreciate links to things I may have
> > missed.
> >
> > Thanks!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message