spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <>
Subject [jira] [Commented] (SPARK-5439) Expose yarn app id for yarn mode
Date Tue, 27 Jan 2015 22:59:34 GMT


Marcelo Vanzin commented on SPARK-5439:

Hi bc, can you provide more context here? The {{SparkListenerApplicationStart}} event contains
the app ID. I can see two issues with that:

- In client mode, that event is sent before the user code has a chance to register any listeners.
This would be covered by SPARK-5190 and SPARK-5411.
- In cluster mode, there's no API to query. The only way at all, right now, would be to parse
the stderr of the process doing the launching and look for the id.

> Expose yarn app id for yarn mode
> --------------------------------
>                 Key: SPARK-5439
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: YARN
>    Affects Versions: 1.0.0
>            Reporter: bc Wong
> When submitting Spark apps on YARN, the caller should be able to get back the YARN app
ID programmatically. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message