hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-8257) Native service should automatically adding escapes for environment/launch cmd before sending to YARN
Date Mon, 07 May 2018 23:57:00 GMT
Wangda Tan created YARN-8257:
--------------------------------

             Summary: Native service should automatically adding escapes for environment/launch
cmd before sending to YARN
                 Key: YARN-8257
                 URL: https://issues.apache.org/jira/browse/YARN-8257
             Project: Hadoop YARN
          Issue Type: Bug
          Components: yarn-native-services
            Reporter: Wangda Tan
            Assignee: Gour Saha


Noticed this issue while using native service: 

Basically, when a string for environment / launch command contains chars like ", /, `: it
needs to be escaped twice.

The first time is from json spec, because of json accept double quote only, it needs an escape.

The second time is from launch container, what we did for command line is: (ContainerLaunch.java)
{code:java}
line("exec /bin/bash -c \"", StringUtils.join(" ", command), "\"");{code}
And for environment:
{code:java}
line("export ", key, "=\"", value, "\"");{code}
An example of launch_command: 
{code:java}
"launch_command": "export CLASSPATH=\\`\\$HADOOP_HDFS_HOME/bin/hadoop classpath --glob\\`"{code}
And example of environment:
{code:java}
"TF_CONFIG" : "{\\\"cluster\\\": {\\\"master\\\": [\\\"master-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"],
\\\"ps\\\": [\\\"ps-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"worker\\\":
[\\\"worker-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"]}, \\\"task\\\": {\\\"type\\\":\\\"${COMPONENT_NAME}\\\",
\\\"index\\\":${COMPONENT_ID}}, \\\"environment\\\":\\\"cloud\\\"}",{code}

To improve usability, I think we should auto escape the input string once. (For example, if
user specified 
{code}
"TF_CONFIG": "\"key\""
{code}
We will automatically escape it to:
{code}
"TF_CONFIG": \\\"key\\\"
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message