hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3805) MR AM not respecting MaxReduceRampUpLimit
Date Fri, 03 Feb 2012 23:53:53 GMT
MR AM not respecting MaxReduceRampUpLimit

                 Key: MAPREDUCE-3805
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3805
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mr-am, mrv2
    Affects Versions: 0.23.0
            Reporter: Vinod Kumar Vavilapalli
             Fix For: 0.23.1

While running GridMixV3 with high memory reduces, we ran into issues where for jobs with significant
number of maps and reduces, when the map progress hits 98-99% but still there are maps pending,
reduces get every new container that RM allocates. And the job takes much longer time than
with usual reduces.

For addressing precisely these issues, a configurable limit was introduced to limit the reduce
ramp up. Unfortunately this limit is not working correctly.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message