hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Ryan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-6858) Enable rotateable JVM garbage collection logs for Hadoop daemons
Date Tue, 13 Jul 2010 21:18:51 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Ryan updated HADOOP-6858:

    Status: Patch Available  (was: Open)

Patch submitted for review. This won't apply, because it needs patching common, mapred, and
hdfs. If there is agreement on how to move forward, I can regenerate different patches for

> Enable rotateable JVM garbage collection logs for Hadoop daemons
> ----------------------------------------------------------------
>                 Key: HADOOP-6858
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6858
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: scripts
>    Affects Versions: 0.22.0
>            Reporter: Andrew Ryan
>         Attachments: HADOOP-6858.patch
> The purpose of this enhancement is to make it easier to collect garbage collection logs
and insure that they persist across restarts in the same way that the standard output files
of Hadoop daemon JVM's currently does.
> Garbage collection logs are a vital debugging tool for administrators and developers.
In our production environments, at some point or another, every single type of Hadoop daemon
has OOM'ed or experienced other significant issues related to GC and/or lack of heap memory.
For the longest time, we have put in garbage collection logs in our HADOOP_NAMENODE_OPTS,
HADOOP_JOBTRACKER_OPTS, etc. by using options like "-XX:+PrintGCDateStamps -XX:+PrintGCDetails
> Unfortunately, these logs don't survive a restart of the node, so if a node OOM's and
then is restarted automatically, or manually by someone who is unaware, we lose the GC logs
forever. We also have to manually add GC log options to each daemon. This patch:
> 1) Creates a single, optional, off by default, parameter for specifying GC logging.
> 2) If that parameter is set, automatically enables GC logging for all daemons in the
cluster. The parameter is flexible enough to allow for the different ways various vendor's
JVM's require garbage collection logging to be specified. 
> 3) If GC logging is on, insures that the GC log files for each daemon are rotated with
up to 5 copies kept, same as the .out files currently.
> We are currently running a variation of this patch in our 0.20 install. This patch actually
includes changes to common, mapred, and hdfs, so it obviously cannot be applied as-is, but
is included here for review and comments.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message