spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunanmcg...@gmail.com>
Subject Re: Config properties broken in master
Date Sat, 18 Jan 2014 19:21:08 GMT
change spark.speculation to spark.speculation.switch?

maybe we can restrict that all properties in Spark should be "three levels"


On Sat, Jan 18, 2014 at 2:10 PM, Mridul Muralidharan <mridul@gmail.com>wrote:

> Hi,
>
>   Unless I am mistaken, the change to using typesafe ConfigFactory has
> broken some of the system properties we use in spark.
>
> For example: if we have both
> -Dspark.speculation=true -Dspark.speculation.multiplier=0.95
> set, then the spark.speculation property is dropped.
>
> The rules of parseProperty actually document this clearly [1]
>
>
> I am not sure what the right fix here would be (other than replacing
> use of config that is).
>
> Any thoughts ?
> I would vote -1 for 0.9 to be released before this is fixed.
>
>
> Regards,
> Mridul
>
>
> [1]
> http://typesafehub.github.io/config/latest/api/com/typesafe/config/ConfigFactory.html#parseProperties%28java.util.Properties,%20com.typesafe.config.ConfigParseOptions%29
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message