hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5124) AM lacks flow control for task events
Date Tue, 02 Apr 2013 14:31:19 GMT
Jason Lowe created MAPREDUCE-5124:

             Summary: AM lacks flow control for task events
                 Key: MAPREDUCE-5124
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5124
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mr-am
    Affects Versions: 0.23.5, 2.0.3-alpha
            Reporter: Jason Lowe

The AM does not have any flow control to limit the incoming rate of events from tasks.  If
the AM is unable to keep pace with the rate of incoming events for a sufficient period of
time then it will eventually exhaust the heap and crash.  MAPREDUCE-5043 addressed a major
bottleneck for event processing, but the AM could still get behind if it's starved for CPU
and/or handling a very large job with tens of thousands of active tasks.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message