hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-3340) Deprecate Job.setJobSetupCleanupNeeded()
Date Thu, 03 Nov 2011 13:47:32 GMT
Deprecate Job.setJobSetupCleanupNeeded()

                 Key: MAPREDUCE-3340
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3340
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 0.23.0
            Reporter: Vinod Kumar Vavilapalli
            Assignee: Vinod Kumar Vavilapalli
             Fix For: 0.23.1

We should deprecate the setJobSetupCleanupNeeded() API. It was originally added for performance
reasons to avoid launching new JVMs altogether for job-setup and job-cleanup. With Yarn and
MRAppMaster, setup and cleanup are run inside the AM itself and so nothing much can be gained
by making them optional.

Before 0.23, we could disable set up and cleanup, yet obtain the output when using FileOutputCommitter
in the job-output directory. But post 0.23.0, that won't be the case because of the nested
temporary directories to support AM recoverability. So it makes sense to *not* have cleanupJob

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message