spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Garlapati, Suryanarayana (Nokia - IN/Bangalore)" <suryanarayana.garlap...@nokia.com>
Subject RE: [K8S] Driver and Executor Logging
Date Sat, 08 Sep 2018 16:03:00 GMT
Hi,
Provide the following options in spark-defaults.conf and make sure the log4j.properties file
is available in driver and executor:

spark.driver.extraJavaOptions -Dlog4j.configuration=file:<path>/log4j.properties
spark.executor.extraJavaOptions -Dlog4j.configuration=file: <path>/log4j.properties


Regards
Surya

From: Rohit Menon <rohit.menon2611@gmail.com>
Sent: Friday, September 7, 2018 11:02 PM
To: user@spark.apache.org
Subject: [K8S] Driver and Executor Logging

Hello All,

We are trying to use a custom appender for Spark driver and executor pods. However, the changes
to log4j.properties file in the spark container image are not taking effect. We even tried
simpler changes like changing the logging level to DEBUG.

Has anyone run into similar issues? or successfully changed logging properties for spark on
k8s?

Thanks,
Rohit
Mime
View raw message