spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jamborta <jambo...@gmail.com>
Subject Re: Any issues with repartition?
Date Fri, 03 Oct 2014 09:55:02 GMT
I have two nodes with 96G ram 16 cores, my setup is as follows:

    conf = (SparkConf()
            .setMaster("yarn-cluster")
            .set("spark.executor.memory", "30G")
            .set("spark.cores.max", 32)
            .set("spark.executor.instances", 2)
            .set("spark.executor.cores", 8)
            .set("spark.akka.timeout", 10000)
            .set("spark.akka.askTimeout", 100)
            .set("spark.akka.frameSize", 500)
            .set("spark.cleaner.ttl", 86400)
            .set("spark.tast.maxFailures", 16)
            .set("spark.worker.timeout", 150)

thanks a lot,




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Any-issues-with-repartition-tp13462p15674.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message