hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (YARN-1862) Change mapreduce.jobhistory.done-dir by command line arg seems not working
Date Tue, 01 Apr 2014 21:02:14 GMT

     [ https://issues.apache.org/jira/browse/YARN-1862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jason Lowe resolved YARN-1862.

    Resolution: Invalid

This is a question best asked on the [user@ mailing list|http://hadoop.apache.org/mailing_lists.html#User].
 JIRA is for reporting bugs in the code and tracking new features and is not a user support

mapreduce.jobhistory.done-dir is not a per-job property but rather a property used by the
job history server.  Attempting to override this property in a job will have no effect.  There
currently isn't a way to specify a per-job final destination for job history files.  This
would require some additional support in the job history server as it would need a way to
locate the corresponding job history files for a given job ID if they could be in arbitrary
places for each job.

> Change mapreduce.jobhistory.done-dir by command line arg seems not working
> --------------------------------------------------------------------------
>                 Key: YARN-1862
>                 URL: https://issues.apache.org/jira/browse/YARN-1862
>             Project: Hadoop YARN
>          Issue Type: Bug
>         Environment: Hadoop 2.2.0-cdh5.0.0-beta-2
>            Reporter: Ruirui
>            Priority: Minor
> Basically my requirement is to store the job history file in a customized directory and
maybe history files for job1 in dir1 and job2 in dir2 in YARN. But although I changed "mapreduce.jobhistory.done-dir"
 as command arg and run with the user "mapred" to eliminate permission concerns, the history
files are still generated in the default directory but not in the directory I specified. 
> However if I specify "mapreduce.jobhistory.intermediate-done-dir" in the command line,
it places files in expected directory, only that it is deleted shortly after the job completion
thus not possible for me to get data from the files.
> I searched the Internet and couldn't find any suggestions about this. I am not sure if
it comes from any misconfiguration and I'm not sure if this is the way to ask for help. Would
you please give me a hand? Thank you.

This message was sent by Atlassian JIRA

View raw message