hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5758) Reducer local data is not deleted until job completes
Date Thu, 13 Feb 2014 22:38:19 GMT
Jason Lowe created MAPREDUCE-5758:
-------------------------------------

             Summary: Reducer local data is not deleted until job completes
                 Key: MAPREDUCE-5758
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5758
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 2.2.0, 0.23.10
            Reporter: Jason Lowe


Ran into an instance where a reducer shuffled a large amount of data and subsequently failed,
but the local data is not purged when the task fails but only after the entire job completes.
 This wastes disk space unnecessarily since the data is no longer relevant after the task-attempt
exits.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message