mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: ClassNotFoundException with pseudo/distributed run of KMeans
Date Thu, 16 Jul 2009 10:33:41 GMT
Can you share how you built and how you are running, as in command  
line options, etc.?  Also, JDK version, Hadoop version, etc.

On Jul 16, 2009, at 6:21 AM, Paul Ingles wrote:

> Hi,
>
> Thank you for the suggestion. Unfortunately, when I tried that I  
> received the same error. I've also tried copying the gson jar  
> directly into $HADOOP_HOME/lib (when I was running a single node  
> pseudo-distributed) and get the same error still.
>
> Weirdly enough, if I try and run the Dirichlet example on the  
> cluster I receive another ClassNotFoundException:
>
> 09/07/16 10:27:54 INFO mapred.JobClient: Task Id :  
> attempt_200907161026_0002_m_000001_0, Status : FAILED
> java.lang.RuntimeException: Error in configuring object
>        at  
> org 
> .apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java: 
> 93)
>        at  
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java: 
> 64)
>        at  
> org 
> .apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java: 
> 117)
>        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java: 
> 352)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.reflect.InvocationTargetException
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at  
> sun 
> .reflect 
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at  
> sun 
> .reflect 
> .DelegatingMethodAccessorImpl 
> .invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at  
> org 
> .apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java: 
> 88)
>        ... 5 more
> Caused by: java.lang.RuntimeException: Error in configuring object
>        at  
> org 
> .apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java: 
> 93)
>        at  
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java: 
> 64)
>        at  
> org 
> .apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java: 
> 117)
>        at  
> org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>        ... 10 more
> Caused by: java.lang.reflect.InvocationTargetException
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at  
> sun 
> .reflect 
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at  
> sun 
> .reflect 
> .DelegatingMethodAccessorImpl 
> .invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at  
> org 
> .apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java: 
> 88)
>        ... 13 more
> Caused by: java.lang.RuntimeException:  
> java.lang.ClassNotFoundException:  
> org 
> .apache 
> .mahout 
> .clustering.syntheticcontrol.dirichlet.NormalScModelDistribution
>        at  
> org 
> .apache 
> .mahout 
> .clustering 
> .dirichlet.DirichletMapper.getDirichletState(DirichletMapper.java:95)
>        at  
> org 
> .apache 
> .mahout 
> .clustering.dirichlet.DirichletMapper.configure(DirichletMapper.java: 
> 60)
>        ... 18 more
> Caused by: java.lang.ClassNotFoundException:  
> org 
> .apache 
> .mahout 
> .clustering.syntheticcontrol.dirichlet.NormalScModelDistribution
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:316)
>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java: 
> 288)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>        at  
> org 
> .apache 
> .mahout 
> .clustering 
> .dirichlet.DirichletDriver.createState(DirichletDriver.java:121)
>        at  
> org 
> .apache 
> .mahout 
> .clustering 
> .dirichlet.DirichletMapper.getDirichletState(DirichletMapper.java:71)
>        ... 19 more
>
>
> Hoping this sparks some other suggestions :)
>
> Thanks,
> Paul
>
>
> On Wed Jul 15 22:08:09 UTC 2009, Adil Aijaz <adil@yahoo-inc.com>  
> wrote:
>> try hadoop --config <hod-cluster-dir> jar -libjars <path to gson.jar>
>> <your job/jar file> <your class> <arguments>
>>
>> Adil
>>
>> Paul Ingles wrote:
>>> Hi,
>>>
>>> Apologies for the cross-posting (I also sent this to the Hadoop user
>>> list) but I'm still getting errors if I try and run the KMeans
>>> examples on a cluster, whether that be my single-node Mac Pro, or  
>>> our
>>> cluster. I've attached the stack trace at the bottom of the email.
>>>
>>> The gson jar is definitely included in the packaged .job, and is  
>>> also
>>> in the temporary directory when the task tracker picks up the work.
>>> The gson jar also includes TypeToken.class in the expected path.
>>>
>>> Again, really appreciate people's help in getting this going!
>>>
>>> ----snip----
>>> 09/07/15 17:06:38 INFO mapred.JobClient: Task Id :
>>> attempt_200907151617_0010_m_000000_0, Status : FAILED
>>> java.lang.NoClassDefFoundError: com/google/gson/reflect/TypeToken
>>>    at java.lang.ClassLoader.defineClass1(Native Method)
>>>    at java.lang.ClassLoader.defineClass(ClassLoader.java:703)
>>>    at
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java: 
>>> 124)
>>>    at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
>>>    at java.net.URLClassLoader.access$000(URLClassLoader.java:56)
>>>    at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
>>>    at java.security.AccessController.doPrivileged(Native Method)
>>>    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:319)
>>>    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:330)
>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
>>>    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:402)
>>>    at
>>> org 
>>> .apache 
>>> .mahout.matrix.AbstractVector.asFormatString(AbstractVector.java: 
>>> 374)
>>>
>>>    at
>>> org 
>>> .apache 
>>> .mahout 
>>> .clustering.kmeans.Cluster.outputPointWithClusterInfo(Cluster.java: 
>>> 198)
>>>
>>>    at
>>> org 
>>> .apache 
>>> .mahout 
>>> .clustering 
>>> .kmeans.KMeansClusterMapper.map(KMeansClusterMapper.java:39)
>>>
>>>    at
>>> org 
>>> .apache 
>>> .mahout 
>>> .clustering 
>>> .kmeans.KMeansClusterMapper.map(KMeansClusterMapper.java:32)
>>>
>>>    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>>>    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java: 
>>> 356)
>>>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>>    at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>> Caused by: java.lang.ClassNotFoundException:
>>> com.google.gson.reflect.TypeToken
>>>    at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>>    at java.security.AccessController.doPrivileged(Native Method)
>>>    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:319)
>>>    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:330)
>>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:254)
>>>    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:402)
>>>    ... 20 more
>>> ----snip----
>>>
>>> Incidentally, as part of this work I've also implemented a Pearson
>>> distance measure, if people think it would be useful to be folded in
>>> I'd be happy to get the SVN patch with tests and implementation  
>>> together.
>>>
>>> Thanks,
>>> Paul

--------------------------
Grant Ingersoll
http://www.lucidimagination.com/

Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids)  
using Solr/Lucene:
http://www.lucidimagination.com/search


Mime
View raw message