My Spark Structured Streaming application was performing well for quite some time but all of a sudden from today it has slowed down. I noticed in the Spark UI that the 'No. of Active Tasks' is 1 even though 64 Cores are available. (Please see the attached image).
I don't believe there's any data skew issue related to partitioning of data. What could be the reason for this? Please advise. Thanks.