spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From An Tran <>
Subject Spark Streaming Standalone 1.5 - Stage cancelled because SparkContext was shut down
Date Wed, 30 Sep 2015 00:14:13 GMT
Hello All,

I have several Spark Streaming applications running on Standalone mode in
Spark 1.5.  Spark is currently set up for dynamic resource allocation.  The
issue I am seeing is that I can have about 12 Spark Streaming Jobs running
concurrently.  Occasionally I would see more than half where to fail due
to Stage cancelled because SparkContext was shut down.  It would
automatically restart as it runs on supervised mode.  Attached is the
screenshot of one of the jobs that failed.  Anyone have any insight as to
what is going on?

View raw message