spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From suchenzang <suchenz...@gmail.com>
Subject Spark Streaming Listener to Kill Stages?
Date Thu, 27 Aug 2015 18:14:43 GMT
Hello,

I was wondering if there's a way to kill stages in spark streaming via a
StreamingListener. Sometimes stages will hang, and simply killing it via the
UI:

<http://apache-spark-user-list.1001560.n3.nabble.com/file/n24478/Screen_Shot_2015-08-27_at_11.png>


is enough to let the streaming job continue. Ideally we wouldn't have
hanging issues, but we would rather troubleshoot later via logs instead of
terminating the entire streaming job.

Is there any way to access something that is equivalent to
SparkContext.cancelStage, or is this generally not advised?

Thanks!



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Listener-to-Kill-Stages-tp24478.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message