[ https://issues.apache.org/jira/browse/SPARK-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13955046#comment-13955046
]
wangfei commented on SPARK-1361:
--------------------------------
this is an order issue of construction and starting of the taskScheduler and the dagScheduler
in SparkContext. now taskScheduler start before dagScheduler, so executor backend may register
executor even before dagScheduler(or the eventProcessActor) has not initialized. in this situation
it will throw NPE
> DAGScheduler throws NullPointerException occasionally
> -----------------------------------------------------
>
> Key: SPARK-1361
> URL: https://issues.apache.org/jira/browse/SPARK-1361
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 0.9.0
> Reporter: wangfei
> Fix For: 1.0.0
>
>
> DAGScheduler throws this NullPointerException below occasionally when running spark apps.
> java.lang.NullPointerException
> at org.apache.spark.scheduler.DAGScheduler.executorAdded(DAGScheduler.scala:186)
> at org.apache.spark.scheduler.TaskSchedulerImpl.executorAdded(TaskSchedulerImpl.scala:409)
> at org.apache.spark.scheduler.TaskSchedulerImpl$$anonfun$resourceOffers$1.apply(TaskSchedulerImpl.scala:210)
> at org.apache.spark.scheduler.TaskSchedulerImpl$$anonfun$resourceOffers$1.apply(TaskSchedulerImpl.scala:206)
> at scala.collection.mutable.ArraySeq.foreach(ArraySeq.scala:73)
> at org.apache.spark.scheduler.TaskSchedulerImpl.resourceOffers(TaskSchedulerImpl.scala:206)
> at org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverActor.makeOffers(CoarseGrainedSchedulerBackend.scala:130)
> at org.apache.spark.scheduler.cluster.CoarseGrainedSchedulerBackend$DriverActor$$anonfun$receive$1.applyOrEls
--
This message was sent by Atlassian JIRA
(v6.2#6252)
|