spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luis Ángel Vicente Sánchez <>
Subject Kafka Receiver not recreated after executor died
Date Tue, 16 Dec 2014 16:53:33 GMT
Dear spark community,

We were testing a spark failure scenario where the executor that is running
a Kafka Receiver dies.

We are running our streaming jobs on top of mesos and we killed the mesos
slave that was running the executor ; a new executor was created on another
mesos-slave but according to the Driver UI, the Kafka receiver location is
still on the dead slave.

It seems that if the executor where the receiver was running dies, spark on
top of mesos is not able to create a new receiver on a different executor
to continue working.

Is that a known issue? Is there a way we could ensure that the receiver
would be recreated?

Kind regards,

Luis Vicente

View raw message