spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Deadlock between spark logging and wildfly logging
Date Fri, 28 Nov 2014 17:07:41 GMT
I'm kind of guessing here, but it looks like the log4j class has
locked its own internal data structure (ConsoleAppender) and then has
a lock on System.err. The JBoss logger looks like it swaps out
System.err (?) with its logging stub, and has that locked, and then
the stub routes to log4j to log, which is locked.

I wonder if you can get jboss to not do that, or instead log directly
to log4j. I sort of doubt it. I also think the 'embedded Spark' use
case is technically unsupported.Maybe you can configure log4j to not
log anything from jboss?

On Fri, Nov 28, 2014 at 4:59 PM, Charles <charles.li@cenx.com> wrote:
> Here you go.
>
> "Result resolver thread-3" - Thread t@35654
>    java.lang.Thread.State: BLOCKED
>         at java.io.PrintStream.flush(PrintStream.java:335)
>         - waiting to lock <104f7200> (a java.io.PrintStream) owned by
> "null_Worker-1" t@1022
>         at
> org.jboss.stdio.StdioContext$DelegatingPrintStream.flush(StdioContext.java:216)
>         at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:297)
>         at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
>         - locked <13e0275f> (a java.io.OutputStreamWriter)
>         at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
>         at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
>         at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
>         at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
>         at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
>         - locked <1af95e38> (a org.apache.log4j.ConsoleAppender)
>         at
> org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:42)
>         at org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:79)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:296)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304)
>         at org.jboss.logmanager.Logger.logRaw(Logger.java:721)
>         at org.slf4j.impl.Slf4jLogger.log(Slf4jLogger.java:326)
>         at org.slf4j.impl.Slf4jLogger.log(Slf4jLogger.java:320)
>         at org.slf4j.impl.Slf4jLogger.info(Slf4jLogger.java:180)
>         at org.apache.spark.Logging$class.logInfo(Logging.scala:50)
>
>
> "null_Worker-1" - Thread t@1022
>    java.lang.Thread.State: BLOCKED
>         at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:231)
>         - waiting to lock <1af95e38> (a org.apache.log4j.ConsoleAppender) owned
by
> "Result resolver thread-3" t@35654
>         at
> org.apache.log4j.JBossAppenderHandler.doPublish(JBossAppenderHandler.java:42)
>         at org.jboss.logmanager.ExtHandler.publish(ExtHandler.java:79)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:296)
>         at org.jboss.logmanager.LoggerNode.publish(LoggerNode.java:304)
>         at org.jboss.logmanager.Logger.logRaw(Logger.java:721)
>         at org.jboss.logmanager.Logger.log(Logger.java:506)
>         at
> org.jboss.stdio.AbstractLoggingWriter.write(AbstractLoggingWriter.java:71)
>         - locked <2a11d902> (a java.lang.StringBuilder)
>         at org.jboss.stdio.WriterOutputStream.finish(WriterOutputStream.java:143)
>         at org.jboss.stdio.WriterOutputStream.flush(WriterOutputStream.java:164)
>         - locked <2c765985> (a sun.nio.cs.US_ASCII$Decoder)
>         at java.io.PrintStream.write(PrintStream.java:482)
>         - locked <104f7200> (a java.io.PrintStream)
>         at
> org.jboss.stdio.StdioContext$DelegatingPrintStream.write(StdioContext.java:264)
>         at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
>         at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
>         at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
>         at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
>         - locked <15f65ea5> (a java.io.OutputStreamWriter)
>         at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
>         at java.io.PrintWriter.flush(PrintWriter.java:320)
>         - locked <15f65ea5> (a java.io.OutputStreamWriter)
>         at clojure.core$flush.invoke(core.clj:3429)
>         at taoensso.timbre$str_println.doInvoke(timbre.clj:15)
>         at clojure.lang.RestFn.invoke(RestFn.java:408)
>         at taoensso.timbre$fn__3179.invoke(timbre.clj:170)
>         at clojure.core$juxt$fn__4209.invoke(core.clj:2433)
>         at
> taoensso.timbre$wrap_appender_juxt$fn__3244$fn__3248.invoke(timbre.clj:297)
>         at
> taoensso.timbre$wrap_appender_juxt$fn__3229$fn__3231.invoke(timbre.clj:319)
>         at taoensso.timbre$send_to_appenders_BANG_.doInvoke(timbre.clj:398)
>         at clojure.lang.RestFn.invoke(RestFn.java:866)
>         at
> cenx.levski.performance_exception_calculation$scheduled_exception_calculation.doInvoke(performance_exception_calculation.clj:207)
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Deadlock-between-spark-logging-and-wildfly-logging-tp20009p20013.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message