spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Agostino Calamita <agostino.calam...@gmail.com>
Subject spark.executor.extraJavaOptions inside application code
Date Wed, 02 May 2018 10:59:38 GMT
Hi all,
I wrote an application that needs an environment variable. I can set this
variable with

--conf 'spark.executor.extraJavaOptions=-Dbasicauth=myuser:mypwd'

in spark-submit and it works well in standalone cluster mode.

But, I want set it inside the application code, because the variable
contains a password.

How can I do ?

I tried with:

    SparkSession spark = SparkSession
                  .builder()
                  .appName("Java Spark Solr ETL")
                  .getOrCreate();


spark.sparkContext().conf().setExecutorEnv("spark.executor.extraJavaOptions",
"-Dbasicauth=myuser:mypassword");

but it doesn't work.

Thanks.

Mime
View raw message