spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From geoHeil <>
Subject ThreadPoolExecutor - slow spark job
Date Fri, 23 Dec 2016 13:19:52 GMT

I built a spark job which is very slow.
ThreadPoolExecutor is executed for every second task of my custom spark
pipeline step.

Additionally, I noticed that spark is spending a lot of the time in the
garbage collection and sometimes 0 tasks are launched but still the driver
is waiting

I put it up here
as well with a minimal example of

Looking forward to any input to speed up this spark job.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message