spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <>
Subject [jira] [Updated] (SPARK-1110) Clean up and clarify use of SPARK_HOME
Date Sun, 30 Mar 2014 04:13:23 GMT


Patrick Wendell updated SPARK-1110:

    Reporter: Patrick Wendell  (was: Patrick Cogan)

> Clean up and clarify use of SPARK_HOME
> --------------------------------------
>                 Key: SPARK-1110
>                 URL:
>             Project: Apache Spark
>          Issue Type: Improvement
>            Reporter: Patrick Wendell
>            Assignee: Patrick Cogan
>             Fix For: 1.0.0
> In the spirit of SPARK-929 we should clean up the use of SPARK_HOME and, if possible,
remove it entirely.
> We need to look through what this is used for. One use was allowing applications to run
different versions of Spark in standalone mode. For instance, someone could submit an application
with a custom SPARK_HOME and the Worker would launch an Executor using a different path for
Spark. This use case is not widely used and maybe should just be removed.
> The existing constructors that take SPARK_HOME for this purpose should be deprecated
and we should explain that SPARK_HOME is no longer used for this purpose.
> If there are other legitimate reasons for SPARK_HOME, we can keep it around... we need
to audit the uses of it.

This message was sent by Atlassian JIRA

View raw message