kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Stein <crypt...@gmail.com>
Subject Re: Failover for Zookeeper and Kafka
Date Tue, 10 Sep 2013 00:56:11 GMT
You want to setup a Zookeeper ensemble ( always an odd number of servers, three is often acceptable
) 
http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_zkMulitServerSetup

And use Kafka 0.8 replication http://kafka.apache.org/documentation.html#replication in addition
if you want to mirror the cluster like in another data center you could also use https://cwiki.apache.org/confluence/display/KAFKA/Kafka+mirroring+(MirrorMaker)

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop
********************************************/


On Sep 9, 2013, at 8:39 PM, Xuyen On <xon@ancestry.com> wrote:

> Hi everyone,
> 
> I am trying to setup a Kafka cluster and have a couple of questions about failover.
> 
> Has anyone deployed more than one zookeeper for a single Kafka cluster and have high
availability so if one zookeeper node goes down, the cluster automatically fails over to a
backup zookeeper node? If so, how is this done? 
> 
> My second question is how can I set up for automatic failover if I have a mirror secondary
Kafka cluster. So, if the main Kafka cluster goes down, what do I need to do in order for
my producers and consumers to automatically fail over to the backup mirror Kafka cluster.
Do I need to code this into my producers and consumers, should I setup a DNS redirect in case
the main Kafka cluster goes down to point to the mirror cluster, or is there some other configuration
that I can do?
> 
> Thanks,
> 
> Xuyen
> 
> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message