spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: Programmatically get status of job (WAITING/RUNNING)
Date Tue, 05 Dec 2017 21:41:14 GMT
On Tue, Dec 5, 2017 at 12:43 PM, bsikander <behroz89@gmail.com> wrote:
> 2) If I use context.addSparkListener, I can customize the listener but then
> I miss the onApplicationStart event. Also, I don't know the Spark's logic to
> changing the state of application from WAITING -> RUNNING.

I'm not sure I follow you here. This is something that you are
defining, not Spark.

"SparkLauncher" has its own view of that those mean, and it doesn't match yours.

"SparkListener" has no notion of whether an app is running or not.

It's up to you to define what waiting and running mean in your code,
and map the events Spark provides you to those concepts.

e.g., a job is running after your listener gets an "onJobStart" event.
But the application might have been running already before that job
started.

-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message