spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saliya Ekanayake <esal...@gmail.com>
Subject Spark #cores
Date Wed, 18 Jan 2017 15:26:07 GMT
Hi,

I am running a Spark application setting the number of executor cores 1 and
a default parallelism of 32 over 8 physical nodes.

The web UI shows it's running on 200 cores. I can't relate this number to
the parameters I've used. How can I control the parallelism in a more
deterministic way?

Thank you,
Saliya

-- 
Saliya Ekanayake, Ph.D
Applied Computer Scientist
Network Dynamics and Simulation Science Laboratory (NDSSL)
Virginia Tech, Blacksburg

Mime
View raw message