spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shixiong Zhu <zsxw...@gmail.com>
Subject Re: Setting network variables in spark-shell
Date Mon, 01 Dec 2014 12:26:45 GMT
Don't set `spark.akka.frameSize` to 10000. The max value of
`spark.akka.frameSize` is 2047. The unit is MB.

Best Regards,
Shixiong Zhu

2014-12-01 0:51 GMT+08:00 Yanbo <yanbohappy@gmail.com>:

>
> Try to use spark-shell --conf spark.akka.frameSize=10000
>
> 在 2014年12月1日,上午12:25,Brian Dolan <buddha_314@yahoo.com.INVALID>
写道:
>
> Howdy Folks,
>
> What is the correct syntax in 1.0.0 to set networking variables in spark
> shell?  Specifically, I'd like to set the spark.akka.frameSize
>
> I'm attempting this:
>
> spark-shell -Dspark.akka.frameSize=10000 --executor-memory 4g
>
>
> Only to get this within the session:
>
> System.getProperty("spark.executor.memory")
> res0: String = 4g
> System.getProperty("spark.akka.frameSize")
> res1: String = null
>
>
> I don't believe I am violating protocol, but I have also posted this to
> SO:
> http://stackoverflow.com/questions/27215288/how-do-i-set-spark-akka-framesize-in-spark-shell
>
> ~~~~~~
> May All Your Sequences Converge
>
>
>
>

Mime
View raw message