hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5493) In-memory map outputs can be leaked after shuffle completes
Date Tue, 03 Sep 2013 21:51:52 GMT
Jason Lowe created MAPREDUCE-5493:

             Summary: In-memory map outputs can be leaked after shuffle completes
                 Key: MAPREDUCE-5493
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5493
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 0.23.9, 2.1.0-beta
            Reporter: Jason Lowe

MergeManagerImpl#close adds the contents of inMemoryMergedMapOutputs and inMemoryMapOutputs
to a list of map outputs that is subsequently processed, but it does not clear those sets.
 This prevents some of the map outputs from being garbage collected and significantly reduces
the memory available for the subsequent reduce phase.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message