spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Parhomenko <>
Subject Unable to redirect Spark logs to slf4j
Date Wed, 05 Mar 2014 13:12:13 GMT

I'm trying to redirect Spark logs to slf4j. Spark seem to be using Log4J,
so I did the typical steps of forcing a Log4J-based framework to use slf4j
- manually excluded slf4j-log4j12 and log4j, and included log4j-over-slf4j.
When doing that however Spark starts failing on initialization with:
 at java.lang.ThreadLocal.access$400(
at java.lang.ThreadLocal$ThreadLocalMap.getEntry(
 at java.lang.ThreadLocal$ThreadLocalMap.access$000(
at java.lang.ThreadLocal.get(
 at java.lang.StringCoding.deref(
at java.lang.StringCoding.encode(
 at java.lang.String.getBytes(
at Method)
 at sun.misc.URLClassPath$FileLoader.getResource(
at sun.misc.URLClassPath$FileLoader.findResource(
 at sun.misc.URLClassPath.findResource(
at Method)
at java.lang.ClassLoader.getResource(
 at org.apache.spark.Logging$class.initializeLogging(Logging.scala:109)
at org.apache.spark.Logging$class.initializeIfNecessary(Logging.scala:97)
 at org.apache.spark.Logging$class.log(Logging.scala:36)
at org.apache.spark.util.Utils$.log(Utils.scala:47)
 <last 4 lines repeated many, many times>

There's some related work done in
but it was resolved after 0.9.0 was released. In the last comment Sean
refers to a StackOverflowError which was discussed in the mailing list, I
assume it might be a problem similar to mine but I was not able to find
that discussion.
Is anyone aware of a way to redirect Spark 0.9.0 logs to slf4j?

Best regards,
Sergey Parhomenko

View raw message