spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dean Wampler <deanwamp...@gmail.com>
Subject Re: PriviledgedActionException- Executor error
Date Sun, 03 May 2015 14:17:31 GMT
So, it looks like the Akka's remote call from your master node, where the
CoarseGrainedExecutorBackend is running, to one or more slave nodes is
timing out.

By default, the port is 2552. Do you have a firewall between the nodes in
your cluster that might be blocking this port. If you're not sure, try
logging into your master and running the command "telnet slave_addr 2552"
where "slave_addr" is one of the slave's IP addresses or a routable host
name.

dean

Dean Wampler, Ph.D.
Author: Programming Scala, 2nd Edition
<http://shop.oreilly.com/product/0636920033073.do> (O'Reilly)
Typesafe <http://typesafe.com>
@deanwampler <http://twitter.com/deanwampler>
http://polyglotprogramming.com

On Sun, May 3, 2015 at 4:25 AM, podioss <gregafen@hotmail.com> wrote:

> Hi,
> i am running several jobs in standalone mode and i notice this error in the
> log files in some of my nodes at the start of my jobs:
>
> INFO executor.CoarseGrainedExecutorBackend: Registered signal handlers for
> [TERM, HUP, INT]
> INFO spark.SecurityManager: Changing view acls to: root
> INFO spark.SecurityManager: Changing modify acls to: root
> INFO spark.SecurityManager: SecurityManager: authentication disabled; ui
> acls disabled; users with view permissions: NFO slf4j.Slf4jLogger:
> Slf4jLogger started
> INFO Remoting: Starting remoting
> ERROR security.UserGroupInformation: PriviledgedActionException as:root
> cause:java.util.concurrent.TimeoutException: Futures timed out after [10000
> milliseconds]
> Exception in thread "main" java.lang.reflect.UndeclaredThrowableException:
> Unknown exception in doAs
>         at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1134)
>         at
>
> org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:59)
>         at
>
> org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:115)
>         at
>
> org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:163)
>         at
>
> org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)
> Caused by: java.security.PrivilegedActionException:
> java.util.concurrent.TimeoutException: Futures timed out after [10000
> milliseconds]
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>         ... 4 more
> Caused by: java.util.concurrent.TimeoutException: Futures timed out after
> [10000 milliseconds]
>         at
> scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
>         at
> scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
>         at
> scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)
>         at
>
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
>         at scala.concurrent.Await$.result(package.scala:107)
>         at akka.remote.Remoting.start(Remoting.scala:180)
>         at
> akka.remote.RemoteActorRefProvider.init(RemoteActorRefProvider.scala:184)
>         at akka.actor.ActorSystemImpl.liftedTree2$1(ActorSystem.scala:618)
>         at
> akka.actor.ActorSystemImpl._start$lzycompute(ActorSystem.scala:615)
>         at akka.actor.ActorSystemImpl._start(ActorSystem.scala:615)
>         at akka.actor.ActorSystemImpl.start(ActorSystem.scala:632)
>         at akka.actor.ActorSystem$.apply(ActorSystem.scala:141)
>         at akka.actor.ActorSystem$.apply(ActorSystem.scala:118)
>         at
>
> org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
>         at
> org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:54)
>         at
> org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
>         at
>
> org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1676)
>         at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
>         at
> org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1667)
>         at
> org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:56)
>         at
>
> org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:122)
>         at
>
> org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:60)
>         at
>
> org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:59)
>         ... 7 more
> INFO remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote
> daemon.
>
> These errors result in executor losses at the beginning and i have been
> trying to find a way to solve this with no success, so if anyone has a clue
> please let me know.
>
> Thank you
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/PriviledgedActionException-Executor-error-tp22745.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message