hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4714) Historyserver retrieves job from jobID more often than necessary
Date Mon, 08 Oct 2012 20:20:02 GMT
Jason Lowe created MAPREDUCE-4714:

             Summary: Historyserver retrieves job from jobID more often than necessary
                 Key: MAPREDUCE-4714
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4714
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobhistoryserver, mrv2
    Affects Versions: 2.0.0-alpha, 0.23.3
            Reporter: Jason Lowe
            Priority: Minor

When serving up a web page for job history, AppController.requireJob() is converting from
a jobID to a job twice when it only needs to do it once.  The resulting job is stored into
the app object, which we could, in turn, use in historyserver web page render() methods to
simply retrieve the job rather than performing yet another jobID->job lookup.  That would
cut the job lookups down from 3 to 1.

jobID->job lookups aren't necessarily cheap.  If the job isn't in the historyserver joblist
cache then the historyserver needs to scan directories looking for it, adding an unnecessary
extra load onto the namenode.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message