spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: Can i get callback notification on Spark job completion ?
Date Fri, 28 Oct 2016 18:28:54 GMT
On Fri, Oct 28, 2016 at 11:14 AM, Elkhan Dadashov <elkhan8502@gmail.com> wrote:
> But if the map task will finish before the Spark job finishes, that means
> SparkLauncher will go away. if the SparkLauncher handle goes away, then I
> lose the ability to track the app's state, right ?
>
> I'm investigating if there is a way to know Spark job completion (without
> Spark Job History Server) in asynchronous manner.

Correct. As I said in my other reply to you, if you can't use Spark's
API for whatever reason, you have to talk directly to the cluster
managers, and at that point it's out of Spark's hands to help you.

-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message