spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Percy <>
Subject Re: Spark streaming on YARN?
Date Sat, 11 Jan 2014 01:47:58 GMT
On Fri, Jan 10, 2014 at 6:35 AM, Neal Sidhwaney <> wrote:

> Are you setting the SPARK_JAVA_OPTS environment variable? Under 0.8.1, we
> had to set that to -Dlog4j.configuration=<path to logging properties file>
> to get STDOUT for workers.  This was in standalone mode, though not with
> YARN, so YMMV.
>  Neal

Thanks for the idea Neal. The docs say that just putting it into conf/
should do the trick, but apparently that's not the case. I tried setting
that and it just stopped printing on the shell and had no effect on the
deployed container.

Too much struggling with this, I'm going to work on getting this running in
standalone mode.


View raw message