hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kartik Bhatia (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-7109) On completion of shuffle phase in reducers, mappers should not be launched again
Date Mon, 11 Jun 2018 06:38:00 GMT
Kartik Bhatia created MAPREDUCE-7109:
----------------------------------------

             Summary: On completion of shuffle phase in reducers, mappers should not be launched
again
                 Key: MAPREDUCE-7109
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7109
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
            Reporter: Kartik Bhatia


Currently, in actOnUnusableNodes in JobImpl mappers are re launched irrespective of whether
their output is further required or not. This leads to unnecessary compute. Idea is to keep
track of all shuffle phases of reducer and not relaunch mapper if shuffle phase in reducers
is already done



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org


Mime
View raw message