nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki ...@getopt.org>
Subject Re: Upgrading to Hadoop 0.22.0+
Date Tue, 13 Dec 2011 16:48:32 GMT
On 13/12/2011 17:42, Lewis John Mcgibbney wrote:
> Hi Markus,
>
> I'm certainly in agreement here. If you like to open a Jira, we can
> begin the build up a picture of what is required.
>
> Lewis
>
> On Tue, Dec 13, 2011 at 4:41 PM, Markus Jelsma
> <markus.jelsma@openindex.io>  wrote:
>> Hi,
>>
>> To keep up with the rest of the world i believe we should move from the old
>> Hadoop mapred API to the new MapReduce API, which has already been done for
>> the nutchgora branch. Upgrading from hadoop-core to hadoop-common is easily
>> done in Ivy but all jobs must be tackled and we have many jobs!
>>
>> Anyone to give pointers and helping hand in this large task?

I guess the question is also whether the 0.22 is compatible enough to 
compile more or less with the existing code that uses the old api. If it 
does, then we can do the transition gradually, if it doesn't then it's a 
bigger issue.

This is easy to verify - just drop in the 0.22 jars and see if it 
compiles / tests are passing.

-- 
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Mime
View raw message