spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CodingCat <>
Subject [GitHub] incubator-spark pull request: [SPARK-1092] remove SPARK_MEM usage ...
Date Fri, 14 Feb 2014 23:33:58 GMT
GitHub user CodingCat opened a pull request:

    [SPARK-1092] remove SPARK_MEM usage in sparkcontext.scala
    Currently, users will usually set SPARK_MEM to control the memory usage of driver programs,
(in spark-class)
    92 JAVA_OPTS="$JAVA_OPTS -Djava.library.path=$SPARK_LIBRARY_PATH"
    if they didn't set spark.executor.memory, the value in this environment variable will
also affect the memory usage of executors, because the following lines in SparkContext
    privatespark val executorMemory = conf.getOption("spark.executor.memory")
    since SPARK_MEM has been (proposed to) deprecated in SPARK-929 (
and the corresponding PR (
    we should remove this line

You can merge this pull request into a Git repository by running:

    $ git pull clean_spark_mem

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #602
commit 31ea1cfe4231c7ff14f2721fa2be99baa43c29d0
Author: CodingCat <>
Date:   2014-02-14T23:31:03Z

    remove SPARK_MEM usage in sparkcontext.scala


View raw message