spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Martin (JIRA)" <>
Subject [jira] [Commented] (SPARK-6305) Add support for log4j 2.x to Spark
Date Wed, 22 Aug 2018 19:46:00 GMT


Chris Martin commented on SPARK-6305:

Thanks [~srowen] and [] for the feedback.

So far my strategy has been to exclude log4j 1.x (log4j:log4j) and the log4j 1.x slf4j binding
(org.slf4j.slf4j-log4j12) from transitive dependencies.  In their place I'm adding in the
in the log4j-1.2-api bridge which should provide the log4j 1.x classes they expect with the
output redirected to  log4j2.  Hopefully this should avoid the stacktrace issue that Steve
mentions but that would depend if any of the dependencies are doing anything funky. 

The only problems I forsee with this are:

1) There a bunch of stuff going on in logging.scala to do with lifecycle management and potential
use of JUL that I'm genuinely unsure as to what it's trying to achieve.  I might have to
ask on the developer mailing list to find out what's going on here, but if anyone here understands
then do let me know.  From what I've seen there's no need to shade any of this, but it's
perfectly possible I might be missing something.

2) I'm less familiar with the projects in external- and I'm not entirely sure under what environments
they should run.  I'm going to leave these til the end when hopefully I'll understand this
a bit more!

3)  As has been mentioned- if and when we decide to move to log4j2- everyone's existing properties
files will need to change (and from what I've read on the log4j jira- they will never have
perfect backwards compatibility).  For now I'm just seeing if we can make spark use log4j2.






> Add support for log4j 2.x to Spark
> ----------------------------------
>                 Key: SPARK-6305
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>            Reporter: Tal Sliwowicz
>            Priority: Minor
> log4j 2 requires replacing the slf4j binding and adding the log4j jars in the classpath.
Since there are shaded jars, it must be done during the build.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message