spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Pfeiffer <...@preferred.jp>
Subject Re: How to get the master URL at runtime inside driver program?
Date Tue, 20 Jan 2015 00:48:33 GMT
Hi,

On Sun, Jan 18, 2015 at 11:08 AM, guxiaobo1982 <guxiaobo1982@qq.com> wrote:
>
> Driver programs submitted by the spark-submit script will get the runtime
> spark master URL, but how it get the URL inside the main method when
> creating the SparkConf object?
>

The master will be stored in the spark.master property. I use the following
snippet:

    // When run through spark-submit, the Java system property
"spark.master"
    // will contain the master passed to spark-submit and we *must* use the
    // same; otherwise use "local[3]".
    val master = scala.util.Properties.propOrElse("spark.master",
"local[3]")

Tobias

Mime
View raw message