nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki ...@getopt.org>
Subject Re: Upgrading to Hadoop 0.22.0+
Date Tue, 13 Dec 2011 17:57:48 GMT
On 13/12/2011 18:04, Markus Jelsma wrote:
> Hi
>
> I did a quick test to see what happens and it won't compile. It cannot find
> our old mapred API's in 0.22. I've also tried 0.20.205.0 which compiles but
> won't run and many tests fail with stuff like.
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/codehaus/jackson/map/JsonMappingException
>          at
> org.apache.nutch.util.dupedb.HostDeduplicator.deduplicator(HostDeduplicator.java:421)

Hmm... what's that? I don't see this class (or this package) in the 
Nutch tree. Also, trunk doesn't use JSON for anything as far as I know.

>          at
> org.apache.nutch.util.dupedb.HostDeduplicator.run(HostDeduplicator.java:443)
>          at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>          at
> org.apache.nutch.util.dupedb.HostDeduplicator.main(HostDeduplicator.java:431)
> Caused by: java.lang.ClassNotFoundException:
> org.codehaus.jackson.map.JsonMappingException
>          at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>          at java.security.AccessController.doPrivileged(Native Method)
>          at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>          at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>          at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>          at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>          ... 4 more
>
> I think this can be overcome but we cannot hide from the fact that all jobs
> must be ported to the new API at some point.
>
> You did some work on the new API's, did you come across any cumbersome issues
> when working on it?

It was quite some time ago .. but I don't remember anything being really 
complicated, it was just tedious - and once you've done one class the 
other classes follow roughly the same pattern.


-- 
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Mime
View raw message