spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Nnamdi Akalugwu <cprenzb...@gmail.com>
Subject Re: How Spark HA works
Date Sat, 20 Aug 2016 03:56:08 GMT
I am experiencing this exact issue. Does anyone know what's going on with
the zookeeper setup?

On Jul 5, 2016 10:34 AM, "Akmal Abbasov" <akmal.abbasov@icloud.com> wrote:
>
> Hi,
> I'm trying to understand how Spark HA works. I'm using Spark 1.6.1 and
Zookeeper 3.4.6.
> I've add the following line to $SPARK_HOME/conf/spark-env.sh
> export SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER
-Dspark.deploy.zookeeper.url=zk1:2181,zk2:2181,zk3:2181
-Dspark.deploy.zookeeper.dir=/spark
> It's working so far.
> I'd like to setup a link which will always go to active master UI(I'm
using Spark in Standalone).
> I've checked the znode /spark, and it contains
> [leader_election, master_status]
> I'm assuming that master_status znode will contain ip address of the
current active master, is it true? Because in my case this znode isn't
updated after failover.
> And how /spark/leader_election works, because it doesn't contain any data.
> Thank you.
>
> Regards,
> Akmal
>
>

Mime
View raw message