Is your app accumulating a lot of streaming state? that's one reason something could slow down after a long time. Some memory leak in your app putting GC/memory pressure on the JVM, etc too.

On Thu, Jan 21, 2021 at 5:13 AM Eric Beabes <mailinglists19@gmail.com> wrote:
Hello,

My Spark Structured Streaming application was performing well for quite some time but all of a sudden from today it has slowed down. I noticed in the Spark UI that the 'No. of Active Tasks' is 1 even though 64 Cores are available. (Please see the attached image).

I don't believe there's any data skew issue related to partitioning of data. What could be the reason for this? Please advise. Thanks.



---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org