spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Or (JIRA)" <>
Subject [jira] [Commented] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN
Date Sat, 10 May 2014 01:05:32 GMT


Andrew Or commented on SPARK-1755:

You are correct. However, it also doesn't work for yarn-cluster.

> Spark-submit --name does not resolve to application name on YARN
> ----------------------------------------------------------------
>                 Key: SPARK-1755
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 0.9.1
>            Reporter: Andrew Or
>            Assignee: Andrew Or
>            Priority: Blocker
>             Fix For: 1.0.0
> In YARN client mode, --name is ignored because the deploy mode is client, and the name
is for some reason a [cluster config|].
> In YARN cluster mode, --name is passed to the org.apache.spark.deploy.yarn.Client as
a command line argument. The Client class, however, uses this name only as the [app name for
the RM|],
but not for Spark. In other words, when SparkConf attempts to load default configs, application
name is not set.
> In both cases, passing --name to SparkSubmit does not actually cause Spark to adopt it
as its application name, despite what the usage promises.

This message was sent by Atlassian JIRA

View raw message