spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luis Ángel Vicente Sánchez <>
Subject Re: Kafka Receiver not recreated after executor died
Date Tue, 16 Dec 2014 16:56:42 GMT
It seems to be slightly related to this:

But in this case, it's not the Task that is failing but the entire executor
where the Kafka Receiver resides.

2014-12-16 16:53 GMT+00:00 Luis Ángel Vicente Sánchez <>:
> Dear spark community,
> We were testing a spark failure scenario where the executor that is
> running a Kafka Receiver dies.
> We are running our streaming jobs on top of mesos and we killed the mesos
> slave that was running the executor ; a new executor was created on another
> mesos-slave but according to the Driver UI, the Kafka receiver location is
> still on the dead slave.
> It seems that if the executor where the receiver was running dies, spark
> on top of mesos is not able to create a new receiver on a different
> executor to continue working.
> Is that a known issue? Is there a way we could ensure that the receiver
> would be recreated?
> Kind regards,
> Luis Vicente

View raw message