spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject options set in is not reflecting on actual execution
Date Thu, 19 Jun 2014 05:21:13 GMT
Hi all,

I have a doubt regarding the options in I set the following values in the file
in master and 2 workers

SPARK_DAEMON_JAVA_OPTS+="- Dspark.akka.timeout=300000 -Dspark.akka.frameSize=10000 -Dspark.blockManagerHeartBeatMs=800000

But SPARK_EXECUTOR_MEMORY is showing 4g in web UI.Do I need to change it anywhere else to
make it 4g and to reflect it in web UI.

A warning is coming that blockManagerHeartBeatMs is exceeding 450000 while executing a process
even though I set it to 800000.

So I doubt whether it should be set  as SPARK_MASTER_OPTS or SPARK_WORKER_OPTS..
Thanks & Regards, 
Meethu M
View raw message