hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gera Shegalov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5773) Provide dedicated MRAppMaster syslog length limit
Date Sun, 02 Mar 2014 07:44:19 GMT
Gera Shegalov created MAPREDUCE-5773:

             Summary: Provide dedicated MRAppMaster syslog length limit
                 Key: MAPREDUCE-5773
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5773
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: mr-am
    Affects Versions: 2.3.0
            Reporter: Gera Shegalov
            Assignee: Gera Shegalov
            Priority: Blocker
             Fix For: 2.4.0

Currently mapreduce.task.userlog.limit.kb controls both the length of task attempt logs and
MA-AM attempt logs. Obviously, MR-AM log is not userlog.
We are interested in keeping MR-AM log either in its entirety or in much larger size than
task logs for debugging. 

MAPREDUCE-5672 introduced CRLA. We already use a dedicated setting for how many backups of
rolled files we keep for MR-AM. However, for large jobs with tens of megabyte AM log, it means
that you have a long series of files. A natural improvement is to have a configuration yarn.app.mapreduce.am.container.log.limit.kb
. It allows to either disable rolling the log for altogether while keeping it for the task
attempts, or to use much a larger limit to make rolling less frequent. 

This message was sent by Atlassian JIRA

View raw message