nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fumio Nakajima (JIRA)" <j...@apache.org>
Subject [jira] [Created] (NUTCH-2637) Number of fetcher reducers is misconfigured when the arg not passed
Date Sat, 18 Aug 2018 06:00:00 GMT
Fumio Nakajima created NUTCH-2637:
-------------------------------------

             Summary: Number of fetcher reducers is misconfigured when the arg not passed
                 Key: NUTCH-2637
                 URL: https://issues.apache.org/jira/browse/NUTCH-2637
             Project: Nutch
          Issue Type: Bug
          Components: fetcher
    Affects Versions: 2.3.1, 2.3
            Reporter: Fumio Nakajima


I'm kind a new to this, so sorry if i'm wrong.
 The thing is the number of fetcher reducers are currently set to the value of "mapred.map.tasks"
when the arg not passed. It should be "mapred.reduce.tasks".

 

Line: 199, branch-2.X
{code:java}
if (numTasks == null || numTasks < 1) {
currentJob.setNumReduceTasks(currentJob.getConfiguration().getInt(
"mapred.map.tasks", currentJob.getNumReduceTasks()));
} else {
currentJob.setNumReduceTasks(numTasks);
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message