hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gera Shegalov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5897) Provide a utility to be able inspect the config as seen by a hadoop client daemon
Date Wed, 21 May 2014 03:55:38 GMT
Gera Shegalov created MAPREDUCE-5897:
----------------------------------------

             Summary: Provide a utility to be able inspect the config as seen by a hadoop
client daemon 
                 Key: MAPREDUCE-5897
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5897
             Project: Hadoop Map/Reduce
          Issue Type: New Feature
            Reporter: Gera Shegalov
            Assignee: Gera Shegalov


To ease debugging of config issues it is convenient to be able to generate a config as seen
by the job client or a hadoop daemon

{noformat}
]$ hadoop org.apache.hadoop.util.ConfigTool -help 
Usage: ConfigTool [ -xml | -json ] [ -loadDefaults ] [ resource1... ]
      if resource contains '/', load from local filesystem
      otherwise, load from the classpath

Generic options supported are
-conf <configuration file>     specify an application configuration file
-D <property=value>            use value for given property
-fs <local|namenode:port>      specify a namenode
-jt <local|jobtracker:port>    specify a job tracker
-files <comma separated list of files>    specify comma separated files to be copied
to the map reduce cluster
-libjars <comma separated list of jars>    specify comma separated jar files to include
in the classpath.
-archives <comma separated list of archives>    specify comma separated archives to
be unarchived on the compute machines.

The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]
{noformat}

{noformat}
$ hadoop org.apache.hadoop.util.ConfigTool -Dmy.test.conf=val mapred-site.xml ./hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/etc/hadoop/core-site.xml
| python -mjson.tool
{
    "properties": [
        {
            "isFinal": false,
            "key": "mapreduce.framework.name",
            "resource": "mapred-site.xml",
            "value": "yarn"
        },
        {
            "isFinal": false,
            "key": "mapreduce.client.genericoptionsparser.used",
            "resource": "programatically",
            "value": "true"
        },
        {
            "isFinal": false,
            "key": "my.test.conf",
            "resource": "from command line",
            "value": "val"
        },
        {
            "isFinal": false,
            "key": "from.file.key",
            "resource": "hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/etc/hadoop/core-site.xml",
            "value": "from.file.val"
        },
        {
            "isFinal": false,
            "key": "mapreduce.shuffle.port",
            "resource": "mapred-site.xml",
            "value": "${my.mapreduce.shuffle.port}"
        }
    ]
}
{noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message