spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Lowenthal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-15564) App name is the main class name in Spark streaming jobs
Date Thu, 26 May 2016 16:30:15 GMT

     [ https://issues.apache.org/jira/browse/SPARK-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steven Lowenthal updated SPARK-15564:
-------------------------------------
    Description: 
I've tried everything to set the app name to something other than the class name of the job,
but spark reports the application name as the class.  This adversely affects the ability to
monitor jobs, we can't have dots in the reported app name. 

{code:title=job.java}
  val defaultAppName = "NDS Transform"
   conf.setAppName(defaultAppName)
   println (s"App Name: ${conf.get("spark.app.name")}")

  ...

  val ssc = new StreamingContext(conf, streamingBatchWindow)

{code}

{code:title=output}
App Name: NDS Transform
{code}

Application ID                            Name
app-20160526161230-0017 (kill)  com.gracenote.ongo.spark.NDSStreamAvro


  was:
I've tried everything to set the app name to something other than the class name of the job,
but spark reports the application name as the class.  This adversely affects the ability to
monitor jobs, we can't have dots in the reported app name. 

{code:title=job.java}
  val defaultAppName = "NDS Transform"
   conf.setAppName(defaultAppName)
   println (s"App Name: ${conf.get("spark.app.name")}")
{code}

{code:title=output}
App Name: NDS Transform
{code}

Application ID                            Name
app-20160526161230-0017 (kill)  com.gracenote.ongo.spark.NDSStreamAvro



> App name is the main class name in Spark streaming jobs
> -------------------------------------------------------
>
>                 Key: SPARK-15564
>                 URL: https://issues.apache.org/jira/browse/SPARK-15564
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.6.1
>            Reporter: Steven Lowenthal
>
> I've tried everything to set the app name to something other than the class name of the
job, but spark reports the application name as the class.  This adversely affects the ability
to monitor jobs, we can't have dots in the reported app name. 
> {code:title=job.java}
>   val defaultAppName = "NDS Transform"
>    conf.setAppName(defaultAppName)
>    println (s"App Name: ${conf.get("spark.app.name")}")
>   ...
>   val ssc = new StreamingContext(conf, streamingBatchWindow)
> {code}
> {code:title=output}
> App Name: NDS Transform
> {code}
> Application ID                            Name
> app-20160526161230-0017 (kill)  com.gracenote.ongo.spark.NDSStreamAvro



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message