spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aakash Basu <aakash.spark....@gmail.com>
Subject Efficient Spark-Submit planning
Date Mon, 11 Sep 2017 21:10:47 GMT
Hi,

Can someone please clarify a little on how should we effectively calculate
the parameters to be passed over using spark-submit.

Parameters as in -

Cores, NumExecutors, DriverMemory, etc.

Is there any generic calculation which can be done over most kind of
clusters with different sizes from small 3 node to 100s of nodes.

Thanks,
Aakash.

Mime
View raw message