spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From TJ Klein <>
Subject Spark Standalone Scheduling
Date Wed, 19 Nov 2014 22:46:57 GMT

I am running some Spark code on my cluster in standalone mode. However, I
have noticed that the most powerful machines (32 cores, 192 Gb mem) hardly
get any tasks, whereas my small machines (8 cores, 128 Gb mem) all get
plenty of tasks. The resources are all displayed correctly in the WebUI and
machines all have the same configuration. When 'slaves' is to only contain
the powerful machines they work well, though. However, I would like to make
use of 'all' machines.
Any idea what could be the reason? Or how the scheduler decides on which
machine the task is assigned to?
Would appreciate some help,

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message