spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Costa <igorco...@apache.org>
Subject Re: SPARK_WORKER_MEMORY in Spark Standalone - conf.getenv vs System.getenv?
Date Tue, 16 Feb 2016 22:44:32 GMT
Actually answering the first question:

Is there a reason to use conf to read SPARK_WORKER_MEMORY not
System.getenv as for the other env vars?

You can use the properties file to change the amount, System.getenv would
be bad when you have for example other things running on the JVM which will
cause conflict on some parts.
 Defined usage in properties files is more convenience for custom UI to be
made available.

On Sat, Feb 13, 2016 at 8:38 PM, Sean Owen <sowen@cloudera.com> wrote:

> Yes you said it is only set in a props file, but why do you say that?
> because the resolution of your first question is that this is not
> differently handled.
>
> On Fri, Feb 12, 2016 at 11:11 PM, Jacek Laskowski <jacek@japila.pl> wrote:
> > On Fri, Feb 12, 2016 at 11:08 PM, Sean Owen <sowen@cloudera.com> wrote:
> >> I think that difference in the code is just an oversight. They
> >> actually do the same thing.
> >
> > Correct. Just meant to know the reason if there was any.
> >
> >> Why do you say this property can only be set in a file?
> >
> > I said that conf/spark-defaults.conf can *not* be used to set
> > spark.worker.ui.port property and wondered why is so? It'd be nice to
> > have it for settings (not use workarounds like
> > SPARK_WORKER_OPTS=-Dspark.worker.ui.port=21212). Just spot it and
> > thought I'd ask if it needs to be cleaned up or improved.
> >
> > Jacek
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
View raw message