hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Zimmerman (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5058) Job History files not copied
Date Tue, 12 Mar 2013 17:03:12 GMT
Aaron Zimmerman created MAPREDUCE-5058:

             Summary: Job History files not copied
                 Key: MAPREDUCE-5058
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5058
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobtracker
    Affects Versions: 1.0.4
            Reporter: Aaron Zimmerman

When a job completes, instead of moving the job history files to the DONE folder, an error
is thrown:
2013-03-12 15:03:52,111 ERROR org.apache.hadoop.mapred.JobHistory: Unable to move history
file to DONE canonical subfolder.
java.io.FileNotFoundException: File </jobhistory/.....> does not exist.

I've confirmed that this file does exist, and can be pulled out of HDFS with hadoop fs -get.

I see this issue reported and resolved on other releases, such as: https://issues.apache.org/jira/browse/MAPREDUCE-2463

I believe the issue is that the move always tries to copy the folder from the local file system,
regardless of whether or not the hadoop.job.history.location is in hdfs. This is in moveToDone()
of org/apache/hadoop/mapred/JobHistory.java.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message