nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki>
Subject Re: [jira] [Created] (NUTCH-1225) Migrate CrawlDBScanner to MapReduce API
Date Thu, 15 Dec 2011 12:18:35 GMT
On 15/12/2011 13:13, Markus Jelsma wrote:
> hmm, i don't see how i can use the old mapred MapOutputFormat API with the new
> Job API. job.setOutputFormatClass(MapFileOutputFormat.class) expects an the
> mapreduce.lib.output.MapFileOutputFormat class and won't accept the old API.
> setOutputFormatClass(java.lang.Class<? extends
> org.apache.hadoop.mapreduce.OutputFormat>) in org.apache.hadoop.mapreduce.Job
> cannot be applied to
> (java.lang.Class<org.apache.hadoop.mapred.MapFileOutputFormat>)
> In short, i don't know how i can migrate jobs to the new API on 0.20.x without
> having MapFileOutputFormat present in the new API. Trying to set to old
> mapoutputformat

Ah, no, that's now what I meant ... of course you need to change the 
code to use the new api, and the new code will look quite different :) 
my point was only that it is different in a consistent way, so after 
you've ported one or two classes the other ones are easy to convert, too...

I'm bogged with other work now, but I'll see if I can prepare an example 
later today...

Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration  Contact: info at sigram dot com

View raw message