spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From elyast <>
Subject Re: little confused about SPARK_JAVA_OPTS alternatives
Date Tue, 01 Jul 2014 04:09:38 GMT
Hi Andrew,

I'm actually using spark-submit, and I tried using
spark.executor.extraJavaOpts to configure tachyon client to connect to
Tachyon HA master, however the configuration settings were not picked up.

On the other hand when I set the same tachyon configuration parameters
through SPARK_JAVA_OPTS or /conf/java_opts it actually worked.

IMHO tachyon client classes are loaded into jvm and since they are mostly
singletons system properties are not being refreshed.

Let me know if you need more info. I have logs from both runs and I can try
different settings on my spark cluster (I am running spark on mesos in fine
grained mode)

Best regards

View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message