spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ÐΞ€ρ@Ҝ (๏̯͡๏) <>
Subject Share your cluster & run details
Date Sun, 28 Jun 2015 18:59:03 GMT
I would like to get a sense of spark YARN cluster used around and this
thread can help others as well

1. Number of nodes in cluster
2. Container memory limit
3. Typical Hardware configuration of worker nodes
4. Typical number of executors used ?
5.  Any other related info you want to share.

How do you decide on number of executors/cores/memory given you know the
amount of data you will process with/without cache enabled.


View raw message