hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-4850) Job recovery may fail if staging directory has been deleted
Date Wed, 09 Jan 2013 15:04:12 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tom White resolved MAPREDUCE-4850.
----------------------------------

       Resolution: Fixed
    Fix Version/s: 1.2.0
     Hadoop Flags: Reviewed

I ran test-patch and it came back clean. I just committed this.
                
> Job recovery may fail if staging directory has been deleted
> -----------------------------------------------------------
>
>                 Key: MAPREDUCE-4850
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4850
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv1
>    Affects Versions: 1.1.1
>            Reporter: Tom White
>            Assignee: Tom White
>             Fix For: 1.2.0
>
>         Attachments: MAPREDUCE-4850.patch, MAPREDUCE-4850.patch
>
>
> The job staging directory is deleted in the job cleanup task, which happens before the
job-info file is deleted from the system directory (by the JobInProgress garbageCollect()
method). If the JT shuts down between these two operations, then when the JT restarts and
tries to recover the job, it fails since the job.xml and splits are no longer available.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message