spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From WangTaoTheTonic <barneystin...@aliyun.com>
Subject Re: Who manage the log4j appender while running spark on yarn?
Date Tue, 23 Dec 2014 02:41:23 GMT
After some discussions with Hadoop guys, I got how the mechanism works.
If we don't add -Dlog4j.configuration into java options to the container(AM
or executors), they will use log4j.properties(if any) under container's
classpath(extraClasspath plus yarn.application.classpath).

If we wanna custom our log4j configuration, we should add
"spark.executor.extraJavaOptions=-Dlog4j.configuration=/path/to/log4j.properties"
or
"spark.yarn.am.extraJavaOptions=-Dlog4j.configuration=/path/to/log4j.properties"
in spark-defaults.conf file.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Who-manage-the-log4j-appender-while-running-spark-on-yarn-tp20778p20818.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message