mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benson Margulies <bimargul...@gmail.com>
Subject Re: Class loader heck in 0.4 with SequenceFileTokenizerMapper.java
Date Thu, 07 Apr 2011 20:13:36 GMT
I'm certain that we're not having this problem in this case. For one
thing, unpacking lucene into the 'main' part of the job jar makes this
problem go away.

On Thu, Apr 7, 2011 at 4:02 PM, Lance Norskog <goksron@gmail.com> wrote:
> Sometimes the 'class not found' exception means that it found the
> named class, but *that* class has missing dependencies. I haven't
> learned how to distinguish "I can't find X" and "I can't find
> dependencies for X".
>
> On Thu, Apr 7, 2011 at 12:37 PM, Benson Margulies <bimargulies@gmail.com> wrote:
>> We've set up a hadoop job jar file to do vectorization.
>>
>> The structure we have is that our code, which is a single class with
>> the main function, is in the unpacked part of the jar file, and
>> mahout-core and its dependencies are in the lib directory of the jar
>> file.
>>
>> The results are the following. In other words, the context class
>> loader, which is what SequenceFileTokenizerMapper uses, doesn't
>> include the lucene-core jar (or, I presume, any of the other jars in
>> the lib dir).
>>
>> Is this old news of some kind? And ideas, other than to unpack all
>> those jars into the unpacked part of our job?
>>
>>
>> Error: java.lang.ClassNotFoundException:
>> org.apache.lucene.analysis.standard.StandardAnalyzer
>>    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>    at java.security.AccessController.doPrivileged(Native Method)
>>    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>    at java.lang.ClassLoader.defineClass1(Native Method)
>>    at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
>>    at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
>>    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>>    at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>>    at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>>    at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>>    at java.security.AccessController.doPrivileged(Native Method)
>>    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>    at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>    at org.apache.mahout.vectorizer.document.SequenceFileTokenizerMapper.setup(SequenceFileTokenizerMapper.java:57)
>>    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
>>    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646)
>>    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
>>    at org.apache.hadoop.mapred.Child$4.run(Child.java:240)
>>    at java.security.AccessController.doPrivileged(Native Method)
>>    at javax.security.auth.Subject.doAs(Subject.java:396)
>>    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
>>    at org.apache.hadoop.mapred.Child.main(Child.java:234)
>>
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>

Mime
View raw message