spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From suchenzang <>
Subject Spark Streaming Listener to Kill Stages?
Date Thu, 27 Aug 2015 18:14:43 GMT

I was wondering if there's a way to kill stages in spark streaming via a
StreamingListener. Sometimes stages will hang, and simply killing it via the


is enough to let the streaming job continue. Ideally we wouldn't have
hanging issues, but we would rather troubleshoot later via logs instead of
terminating the entire streaming job.

Is there any way to access something that is equivalent to
SparkContext.cancelStage, or is this generally not advised?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message