spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Only one Active task in Spark Structured Streaming application
Date Thu, 21 Jan 2021 11:25:00 GMT
Hi,

I'd look at stages and jobs as it's possible that the only task running is
the missing one in a stage of a job. Just guessing...

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
"The Internals Of" Online Books <https://books.japila.pl/>
Follow me on https://twitter.com/jaceklaskowski

<https://twitter.com/jaceklaskowski>


On Thu, Jan 21, 2021 at 12:19 PM Eric Beabes <mailinglists19@gmail.com>
wrote:

> Hello,
>
> My Spark Structured Streaming application was performing well for quite
> some time but all of a sudden from today it has slowed down. I noticed in
> the Spark UI that the 'No. of Active Tasks' is 1 even though 64 Cores are
> available. (Please see the attached image).
>
> I don't believe there's any data skew issue related to partitioning of
> data. What could be the reason for this? Please advise. Thanks.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org

Mime
View raw message