spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Erlandson (JIRA)" <>
Subject [jira] [Commented] (SPARK-1781) Generalized validity checking for configuration parameters
Date Sat, 10 May 2014 21:58:41 GMT


Erik Erlandson commented on SPARK-1781:

Ideally, pre-fab predicates could be defined on a per-parameter basis, that could live somewhere,
either serialized in a file, or in the code, etc.    For example, from SPARK-1779, memoryFraction
should always be checked for being on [0.0, 1.0].  A caller of getDouble() should not have
to specify that if they're asking for memoryFraction (and perhaps should not be allowed to
override that requirement either)

> Generalized validity checking for configuration parameters
> ----------------------------------------------------------
>                 Key: SPARK-1781
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 1.0.0
>            Reporter: William Benton
>            Priority: Minor
> Issues like SPARK-1779 could be handled easily by a general mechanism for specifying
whether or not a configuration parameter value is valid or not (and then excepting or warning
and switching to a default value if it is not).  I think it's possible to do this in a fairly
lightweight fashion.

This message was sent by Atlassian JIRA

View raw message