This is a known race condition - root cause of SPARK-5681

On Mon, Jul 13, 2015 at 3:35 AM, Juan Rodríguez Hortalá <juan.rodriguez.hortala@gmail.com> wrote:
Hi, 

I have noticed that when StreamingContext.stop is called when no receiver has started yet, then the context is not really stopped. Watching the logs it looks like a stop signal is sent to 0 receivers, because the receivers have not started yet, and then the receivers are started and the context keeps working

15/07/13 12:24:52 INFO ReceiverTracker: Sent stop signal to all 0 receivers
15/07/13 12:24:52 INFO RecurringTimer: Started timer for BlockGenerator at time 1436783092600
15/07/13 12:24:52 INFO BlockGenerator: Started BlockGenerator
15/07/13 12:24:52 INFO BlockGenerator: Started block pushing thread
15/07/13 12:24:52 INFO ReceiverSupervisorImpl: Starting receiver
15/07/13 12:24:52 INFO ActorReceiver: Supervision tree for receivers initialized at:akka://sparkDriver/user/Supervisor0
15/07/13 12:24:52 INFO ActorReceiver: Started receiver worker at:akka://sparkDriver/user/Supervisor0/ReceiverActorFoo
15/07/13 12:24:52 INFO ReceiverSupervisorImpl: Called receiver onStart
15/07/13 12:24:52 INFO ProxyReceiverActor: Starting es.ucm.fdi.sscheck.testing.ProxyReceiverActor Actor[akka://sparkDriver/user/Supervisor0/ReceiverActorFoo#629970050]
15/07/13 12:24:52 INFO ReceiverTracker: Registered receiver for stream 0 from 192.168.56.1:49724

My workaround is using a streaming listener to wait for onReceiverStarted, so I don't call stop before that event has occurred  https://gist.github.com/juanrh/b43cdad2d1250b676794 . This is a minor issue but I think it is a surprising behaviour that maybe should be documented somewhere. 

Greetings, 

Juan