Archit, Are you able to get it to work with 1.0.0? 
I tried the --files suggestion from Marcelo and it just changed logging for my client and the appmaster and executors were still the same.


On Jul 30, 2014, at 9:59 PM, Archit Thakur <> wrote:

Hi Marcelo,

Thanks for your quick comment. This doesn't seem working in 1.0.0. release.

-Archit Thakur.

On Thu, Jul 31, 2014 at 3:18 AM, Marcelo Vanzin <> wrote:
Hi Archit,

Are you using spark-submit? If so, can you try adding the following to
its command line:

  --files /dir/

I tested that locally with the master branch and it works, but I don't
have a 1.0.x install readily available for me to test at the moment.
Handling of SPARK_LOG4J_CONF should also be fixed on master, but I
don't remember what was its status in the 1.0 branch...

On Tue, Jul 29, 2014 at 11:37 PM, Archit Thakur
<> wrote:
> Hi,
> I want to manage logging of containers when I run Spark through YARN. I
> checked there is a environment variable exposed to custom
> Setting SPARK_LOG4J_CONF to "/dir/" should ideally make
> containers use
> "/dir/" file for logging. This doesn't seem working.
> Also, there is a file present in the Spark assembly and I
> guess it is picking that only.
> Quick Google gave me this:
> Is this functnality broken/not working yet?
> Or I m mistaken somewhere?
> Thanks.
> Archit.