spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From map reduced <>
Subject Increasing Executor threadpool
Date Thu, 03 Nov 2016 04:48:33 GMT

I am noticing that when there are N cores per executor, each executor only
starts N threads to process the data (so 1 thread per core). Is there a way
to increase more than N threads, i.e. say N+m threads per core?

So I assigned 7 cores/executor, so I see 7 Active Tasks at all times.

[image: Inline image 1]

And only 7 threads doing all the work:

[image: Inline image 2]

Is there any way to make it atleast 2 threads/core?

P.S.: Long running Streaming job, Standalone 2.0.0 cluster


View raw message