spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Spark Streaming Stuck After 10mins Issue...
Date Sun, 07 Jun 2015 16:23:31 GMT
Which consumer are you using? If you can paste the complete code then may
be i can try reproducing it.

Thanks
Best Regards

On Sun, Jun 7, 2015 at 1:53 AM, EH <easonh@gmail.com> wrote:

> And here is the Thread Dump, where seems every worker is waiting for
> Executor
> #6 Thread 95: sparkExecutor-akka.actor.default-dispatcher-22 (RUNNABLE) to
> be complete:
>
> Thread 41: BLOCK_MANAGER cleanup timer (WAITING)
> Thread 42: BROADCAST_VARS cleanup timer (WAITING)
> Thread 44: shuffle-client-0 (RUNNABLE)
> Thread 45: shuffle-server-0 (RUNNABLE)
> Thread 47: Driver Heartbeater (TIMED_WAITING)
> Thread 48: Executor task launch worker-0 (RUNNABLE)
> Thread 56: threadDeathWatcher-2-1 (TIMED_WAITING)
> Thread 81: sparkExecutor-akka.actor.default-dispatcher-18 (WAITING)
> Thread 95: sparkExecutor-akka.actor.default-dispatcher-22 (RUNNABLE)
> **********
> sun.management.ThreadImpl.dumpThreads0(Native Method)
> sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
> org.apache.spark.util.Utils$.getThreadDump(Utils.scala:1777)
>
> org.apache.spark.executor.ExecutorActor$$anonfun$receiveWithLogging$1.applyOrElse(ExecutorActor.scala:38)
>
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)
>
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)
>
> scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)
>
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:53)
>
> org.apache.spark.util.ActorLogReceive$$anon$1.apply(ActorLogReceive.scala:42)
> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:118)
>
> org.apache.spark.util.ActorLogReceive$$anon$1.applyOrElse(ActorLogReceive.scala:42)
> akka.actor.Actor$class.aroundReceive(Actor.scala:465)
>
> org.apache.spark.executor.ExecutorActor.aroundReceive(ExecutorActor.scala:34)
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
> akka.actor.ActorCell.invoke(ActorCell.scala:487)
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
> akka.dispatch.Mailbox.run(Mailbox.scala:220)
>
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
> scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> **********
> Thread 112: sparkExecutor-akka.actor.default-dispatcher-25 (WAITING)
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Stuck-After-10mins-Issue-tp23189p23190.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message