spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christophe Schmitz <cofcof...@gmail.com>
Subject High availability with zookeeper: worker discovery
Date Fri, 31 Jul 2015 02:41:04 GMT
Hi there,

I am trying to run a 3 node spark cluster where each nodes contains a spark
worker and a spark maser. Election of the master happens via zookeeper.

The way I am configuring it is by (on each node) giving the IP:PORT of the
local master to the local worker, and I wish the worker could autodiscover
the elected master automatically.

But unfortunatly, only the local worker of the elected master registered to
the elected master. Why aren't the other worker getting to connect to the
elected master?

The interessing thing is that if I kill the elected master and wait a bit,
then the new elected master sees all the workers!

I am wondering if I am missing something to make this happens without
having to kill the elected master.

Thanks!


PS: I am on spark 1.2.2

Mime
View raw message