hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-6159) No log of JobHistory found in all logs files
Date Wed, 12 Nov 2014 14:36:33 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jason Lowe resolved MAPREDUCE-6159.
    Resolution: Invalid

The JobHistoryEventHandler is code that runs in the ApplicationMaster rather than the job
history server.  You'll find those log messages in the AM logs of individual jobs which are
either aggregated to HDFS (by default) or left on the nodes the AMs ran on if log aggregation
is disabled.

> No log of JobHistory found in all logs files
> --------------------------------------------
>                 Key: MAPREDUCE-6159
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6159
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 2.2.0
>         Environment: Hadoop-2.2.0
>            Reporter: JasonZhu
> I intend to dig into 'mapreduce.jobhistory.intermediate-done-dir' argument, the position
of which is at `JHAdminConfig:73`, to get some comprehension on history server. This argument
is referenced at `JobHistoryEventHandler.moveToDoneNow()`, where history server moves job
summary file 
> from "$[yarn.app.mapreduce.am.staging-dir]/$[user]/.staging" to "$[mapreduce.jobhistory.intermediate-done-dir]/$[user]".

> The following code snippet in `moveToDoneNow()` will definitely write some logs out to
log file, but I can found no any sign of it in all logs in $HADOOP_LOG_DIR via command `grep
"Copied to done location" *`.
>     if (copied)
>         LOG.info("Copied to done location: " + toPath);
>     else 
>         LOG.info("copy failed");
> Is there anything that I missed?

This message was sent by Atlassian JIRA

View raw message