mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastien Bratieres <sb...@cam.ac.uk>
Subject running Dirichlet example on AEMR
Date Thu, 14 May 2009 22:56:47 GMT
Hi,

Thanks Grant, that did it. I'll figure out later what's going on.

Now I'm able to run the kMeans example on Amazon EMR as Stephen did. I want
to run the Dirichlet example, which I launch with
org.apache.mahout.clustering.syntheticcontrol.dirichlet.Job as the main
class from the mahout-examples-0.2-SNAPSHOT.job.

This fails with
java.lang.NoClassDefFoundError:
org/apache/mahout/clustering/dirichlet/DirichletJob
    at
org.apache.mahout.clustering.syntheticcontrol.dirichlet.Job.runJob(Job.java:80)
    at
org.apache.mahout.clustering.syntheticcontrol.dirichlet.Job.main(Job.java:50)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
    at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
    at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)

DirichletJob is located in the .job file, inside
lib/mahout-core-0.2-SNAPSHOT.jar. But apparently the classloader can't find
it.

One difference between kMeans and Dirichlet is
org.apache.mahout.clustering.syntheticcontrol.kmeans.Job line 74
    JobConf conf = new JobConf(Job.class);
org.apache.mahout.clustering.syntheticcontrol.dirichlet.Job line 80
    JobConf conf = new JobConf(DirichletJob.class);
ie the Dirichlet version uses a job class which is in core, while the kMeans
version uses the currently executing Job class from examples. Is there an
issue with this ?

What should I do to work around this error ? Is the MANIFEST.MF file of the
.job contain a pointer to the /lib directory for the jars there to be
visible by the jar classloader ?

Thanks
Sebastien


2009/5/14 Grant Ingersoll <gsingers@apache.org>

> Try running mvn install from the top level dir first.
>
>
> On May 14, 2009, at 11:22 AM, Sebastien Bratieres wrote:
>
>  Hi,
>>
>> I'd like to walk in the footsteps of Stephen Green running Mahout on EMR.
>>
>> He points out that the fix to issue 118 is needed to do that (I first
>> ran into the file system error too). I'm a first-time Maven user and I
>> don't know how to rebuild the mahout-examples-1.0.job file once I have
>> retrieved revision 765769 from SVN (I use Eclipse). I have tried
>> - highlight mahout-examples project
>> - right-click Run As / Maven package (though I'm not sure at all that
>> Maven package is the right option to use!)
>>
>> but that gives me this error
>> ---
>> [INFO] Scanning for projects...
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Building Mahout examples
>> [INFO]
>> [INFO] Id: org.apache.mahout:mahout-examples:jar:0.2-SNAPSHOT
>> [INFO] task-segment: [package]
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] [resources:resources]
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] Copying 0 resource
>> [INFO] [resources:copy-resources]
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] Copying 3 resources
>> [INFO] [compiler:compile]
>> [INFO] Nothing to compile - all classes are up to date
>> [INFO] [resources:testResources]
>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>> [INFO] Copying 3 resources
>> [ERROR]
>>
>> Transitive dependency resolution for scope: test has failed for your
>> project.
>>
>>
>>
>> Error message: Missing:
>> ----------
>> 1) org.apache.mahout:mahout-core:test-jar:tests:0.2-SNAPSHOT
>>
>>  Try downloading the file manually from the project website.
>>
>>  Then, install it using the command:
>>     mvn install:install-file -DgroupId=org.apache.mahout
>> -DartifactId=mahout-core -Dversion=0.2-SNAPSHOT -Dclassifier=tests
>> -Dpackaging=test-jar -Dfile=/path/to/file
>>
>>  Alternatively, if you host your own repository you can deploy the file
>> there:
>>     mvn deploy:deploy-file -DgroupId=org.apache.mahout
>> -DartifactId=mahout-core -Dversion=0.2-SNAPSHOT -Dclassifier=tests
>> -Dpackaging=test-jar -Dfile=/path/to/file -Durl=[url]
>> -DrepositoryId=[id]
>>
>>  Path to dependency:
>>       1) org.apache.mahout:mahout-examples:jar:0.2-SNAPSHOT
>>       2) org.apache.mahout:mahout-core:test-jar:tests:0.2-SNAPSHOT
>>
>> ----------
>> 1 required artifact is missing.
>>
>> for artifact:
>>  org.apache.mahout:mahout-examples:jar:0.2-SNAPSHOT
>>
>> from the specified remote repositories:
>>  Apache snapshots (http://people.apache.org/maven-snapshot-repository),
>>  maven2-repository.dev.java.net (http://download.java.net/maven/2),
>>  central (http://repo1.maven.org/maven2)
>>
>> Group-Id: org.apache.mahout
>> Artifact-Id: mahout-examples
>> Version: 0.2-SNAPSHOT
>> From file: C:\workspace\mahout\examples\pom.xml
>>
>>
>>
>>
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] For more information, run with the -e flag
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD FAILED
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 6 seconds
>> [INFO] Finished at: Thu May 14 16:58:46 CEST 2009
>> [INFO] Final Memory: 3M/22M
>> [INFO]
>> ------------------------------------------------------------------------
>>
>> ---
>>
>> So again, my goal is to have a new mahout-examples-1.0.job file or
>> equivalent that contains the patch for 118 and will run on EMR. What
>> is the right way to do this ?
>>
>> Thanks
>> Sebastien
>>
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message