spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saliya Ekanayake <>
Subject Spark #cores
Date Wed, 18 Jan 2017 15:26:07 GMT

I am running a Spark application setting the number of executor cores 1 and
a default parallelism of 32 over 8 physical nodes.

The web UI shows it's running on 200 cores. I can't relate this number to
the parameters I've used. How can I control the parallelism in a more
deterministic way?

Thank you,

Saliya Ekanayake, Ph.D
Applied Computer Scientist
Network Dynamics and Simulation Science Laboratory (NDSSL)
Virginia Tech, Blacksburg

View raw message