spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ankur Srivastava <ankur.srivast...@gmail.com>
Subject Error when running spark in debug mode
Date Thu, 29 Jan 2015 22:54:24 GMT
Hi,

When ever I enable DEBUG level logs for my spark cluster, on running a job
all the executors die with the below exception. On disabling the DEBUG logs
my jobs move to the next step.


I am on spark-1.1.0

Is this a known issue with spark?

Thanks
Ankur

2015-01-29 22:27:42,467 [main] INFO  org.apache.spark.SecurityManager -
SecurityManager: authentication disabled; ui acls disabled; users with view
permissions: Set(ubuntu); users with modify permissions: Set(ubuntu)

2015-01-29 22:27:42,478 [main] DEBUG org.apache.spark.util.AkkaUtils - In
createActorSystem, requireCookie is: off

2015-01-29 22:27:42,871
[driverPropsFetcher-akka.actor.default-dispatcher-4] INFO
akka.event.slf4j.Slf4jLogger - Slf4jLogger started

2015-01-29 22:27:42,912
[driverPropsFetcher-akka.actor.default-dispatcher-4] INFO  Remoting -
Starting remoting

2015-01-29 22:27:43,057
[driverPropsFetcher-akka.actor.default-dispatcher-4] INFO  Remoting -
Remoting started; listening on addresses :[akka.tcp://
driverPropsFetcher@10.77.9.155:36035]

2015-01-29 22:27:43,060
[driverPropsFetcher-akka.actor.default-dispatcher-4] INFO  Remoting -
Remoting now listens on addresses: [akka.tcp://
driverPropsFetcher@10.77.9.155:36035]

2015-01-29 22:27:43,067 [main] INFO  org.apache.spark.util.Utils -
Successfully started service 'driverPropsFetcher' on port 36035.

2015-01-29 22:28:13,077 [main] ERROR
org.apache.hadoop.security.UserGroupInformation -
PriviledgedActionException as:ubuntu
cause:java.util.concurrent.TimeoutException: Futures timed out after [30
seconds]

Exception in thread "main" java.lang.reflect.UndeclaredThrowableException:
Unknown exception in doAs

        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1134)

        at
org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:52)

        at
org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:113)

        at
org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:156)

        at
org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)

Caused by: java.security.PrivilegedActionException:
java.util.concurrent.TimeoutException: Futures timed out after [30 seconds]

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:415)

        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)

        ... 4 more

Caused by: java.util.concurrent.TimeoutException: Futures timed out after
[30 seconds]

        at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)

        at
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)

        at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:107)

        at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)

        at scala.concurrent.Await$.result(package.scala:107)

        at
org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:125)

        at
org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:53)

        at
org.apache.spark.deploy.SparkHadoopUtil$$anon$1.run(SparkHadoopUtil.scala:52)

        ... 7 more

Mime
View raw message