spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Percy <mpe...@apache.org>
Subject Re: log4j question
Date Sat, 11 Jan 2014 02:19:35 GMT
Khanderao, I think Shay is talking about the details of how to deploy /
where to put the file, not the directives to put inside the log4j file.

I also believe that the docs are wrong w.r.t. log4j, particularly for the
YARN standalone instructions @
https://spark.incubator.apache.org/docs/latest/running-on-yarn.html where
it says:

  # Configure logging
  $ cp conf/log4j.properties.template conf/log4j.properties

Maybe this works / worked for some versions of Spark and/or YARN, just not
the ones I am using. I'd guess there is some classpath / set-up script
related bug somewhere, or maybe some issue with deploying the file to the
cluster via the distributed cache mechanism. Or maybe I just missed a step
somewhere, but I tried a bunch of things.

Mike


On Fri, Jan 10, 2014 at 1:36 PM, Khanderao kand <khanderao.kand@gmail.com>wrote:

> Shay,
>
> This is much more log4j question than a Spark question.
>
>
> Anyway, Copying the templates would enable root logger at INFO level.
>
> Are you interested in suppressing logs? "However this did not suppress
> any logging. " If you want to suppress logging, trying setting to OFF
>
> try
>
> log4j.rootCategory=OFF
>
> Similarly make jetty logger off too.
>
>
>
> Khanderao
>
>
> On Thu, Jan 9, 2014 at 10:49 AM, Shay Seng <shay@1618labs.com> wrote:
>
>> Hey,
>>
>> In "
>> https://spark.incubator.apache.org/docs/0.8.1/configuration.html#configuring-logging
>> "
>> It states:
>> *"Spark uses log4j <http://logging.apache.org/log4j/> for logging. You
>> can configure it by adding a log4j.properties file in the conf directory.
>> One way to start is to copy the existing log4j.properties.template located
>> there."*
>>
>> I did as specified and added a log4j.properties file in the spark conf
>> directory. However this did not suppress any logging.
>> However if I launch spark from a directory that contains a
>> log4j.properties file, it does get read in.
>>
>> Does the docs need re-writing, or is this a bug?
>>
>> tks,
>> Shay
>>
>>
>>
>

Mime
View raw message