mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Mundi <cmu...@gmail.com>
Subject Fwd: Is anyone running Mahout 0.8 against Hadoop 2.0.5-alpha
Date Wed, 24 Jul 2013 06:44:28 GMT
Is anyone running Mahout 0.8 against Hadoop 2.0.5-alpha?

I think I have a hadoop configuration issue.  I've seen similar discussions
but have not found (or recognized) a solution yet.  Rather than tell you
everything I've tried, here's a simple example:

First, I grab the 0.8 tag and build it:

cd /home/hduser/Mahout
svn co http://svn.apache.org/repos/asf/mahout/tags/mahout-0.8/
export $MAHOUT_HOME=cd /home/hduser/Mahout/mahout-0.8
mvn -DskipTests clean install

The maven build succeeds.
I start a single-cluster instance of hadoop and verify it is up.

Now I attempt a canopy clustering, using the helpfully provided script:

hduser@ubuntu:~/Mahout/mahout-0.8/examples/bin$
./cluster-syntheticcontrol.sh
Please select a number to choose the corresponding clustering algorithm
1. canopy clustering
2. kmeans clustering
3. fuzzykmeans clustering
4. dirichlet clustering
5. meanshift clustering
Enter your choice : 1
ok. You chose 1 and we'll use canopy Clustering
creating work directory at /tmp/mahout-work-hduser
Downloading Synthetic control data
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
Current
                                 Dload  Upload   Total   Spent    Left
Speed
100  281k  100  281k    0     0   598k      0 --:--:-- --:--:-- --:--:--
707k
Checking the health of DFS...
13/07/23 23:01:24 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Found 2 items
drwxr-xr-x   - hduser supergroup          0 2013-07-20 08:24 guten
drwxr-xr-x   - hduser supergroup          0 2013-07-23 22:54 testdata
DFS is healthy...
Uploading Synthetic control data to HDFS
rmr: DEPRECATED: Please use 'rm -r' instead.
13/07/23 23:01:26 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Deleted testdata
13/07/23 23:01:28 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
13/07/23 23:01:30 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Successfully Uploaded Synthetic control data to HDFS
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Running on hadoop, using /usr/local/hadoop/bin/hadoop and
HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop
MAHOUT-JOB:
/home/hduser/Mahout/mahout-0.8/examples/target/mahout-examples-0.8-job.jar
13/07/23 23:01:51 WARN driver.MahoutDriver: No
org.apache.mahout.clustering.syntheticcontrol.canopy.Job.props found on
classpath, will use command-line arguments only
Exception in thread "main" java.lang.NoSuchMethodError:
org.apache.hadoop.util.ProgramDriver.driver([Ljava/lang/String;)V
    at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:194)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)


Hmmm... looking at the pom.xml, I see that Mahout 0.8 calls for hadoop
1.1.2.  I also see that there is no POM for hadoop 2.x.y at
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-core/*

Consulting both of

http://hadoop.apache.org/docs/r1.1.2/api/index.html
http://hadoop.apache.org/docs/r2.0.5-alpha/api/index.html

shows that org.apache.hadoop.util has been seriously reworked for 2.x

Perhaps it is as simple as rewriting the examples to use the new hadoop
API?  I've already been working around some trivial API name changes in
hadoop 2.0, so maybe ProgramDriver just got refactored?

I need the federation in hadoop 2.x so I am hoping someone has been down
this path!

So... Is anyone running Mahout 0.8 against Hadoop 2.x.y ?

I am learning the ecosystem, and I appreciate links to things I may have
missed.

Thanks!

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message