spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "donhoff_h" <>
Subject 回复: Job Error:Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@
Date Fri, 25 Dec 2015 08:28:10 GMT
Hi, Saisai Shao

Many thanks for your reply. I used spark v1.3. Unfortunately I can not change to other version.
As to the frequency, yes, every time when I ran a few jobs simultaneously(ususally above 10
jobs), this would appear. 

Is this related to the cpus or memory? I ran those jobs on a virtual machine which has 2 cores
and 4G memory and with yarn-client mode.

------------------ 原始邮件 ------------------
发件人: "Saisai Shao";<>;
发送时间: 2015年12月25日(星期五) 下午4:15
收件人: "donhoff_h"<>; 
抄送: "user"<>; 
主题: Re: Job Error:Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@

I think SparkContext is thread-safe, you could concurrently submit jobs from different threads,
the problem you hit might not relate to this. Can you reproduce this issue each time when
you concurrently submit jobs, or is it happened occasionally?

BTW, I guess you're using the old version of Spark, it may potentially have concurrency problem,
you could switch to a new version to take a try.


On Fri, Dec 25, 2015 at 2:26 PM, donhoff_h <> wrote:

I wrote some spark jobs and these jobs could ran successfully when I ran them one by one.
But if I ran them concurrently, for example 12 jobs parallel running, I met the following
error. Could anybody tell me what cause this? How to solve it? Many Thanks!

Exception in thread "main" Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@,
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
	at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
	at akka.dispatch.BatchingExecutor$
	at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74)
	at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:110)
	at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73)
	at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
	at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:267)
	at akka.remote.DefaultMessageDispatcher.dispatch(Endpoint.scala:89)
	at akka.remote.EndpointReader$$anonfun$receive$2.applyOrElse(Endpoint.scala:937)
	at akka.remote.EndpointActor.aroundReceive(Endpoint.scala:415)
	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(
View raw message