kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Liu <jiangtao....@zuora.com>
Subject Any suggestion about Kafka/Zookeeper migration.
Date Thu, 04 Jan 2018 04:39:35 GMT
Hi All,

This post here is aimed to ask experience about what did you do migration
`Kafka/zookeeper` ?  :)

All of Kafka/zookeeper are running on AWS, because of some reason, we have
to replace all the existed server (you can simply think we will terminate
the old server and create new server to replace it), which is running with
Kafka/zookeeper.

the Kafka infrastructure :

   - 3 zookeeper with exhibitor running together.
   - 6 brokers.

requirements:

   - terminate all of the 3 zookeeper with exhibitor, and also 6 brokers.
   - launch 5 new zookeepers without exhibitor.
   - launch 6 new brokers.
   - 0 or minimum downtime, try our best no impact for the Kafka client.


so far, I have two prepared solution, but any of them has pros & cons, the
solution A is too much manual operations needed, the solution B is some
development effort required and also need to migration all of the Kafka
client to use the self-developed Kafka client with `fail-over` feature.

*Solution A*: simply speaking, this solution is mainly working on manual
operation to add znode / broker to the existed cluster, when all of the
thing ready, I will step by step to terminate the old one.


​

*Solution B:*  simply speaking, the solution will work like fail-over way,
a new cluster will be launched, and control Kafka client's traffic to the
new cluster, and terminate the old cluster.


​

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message