kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Otto <o...@wikimedia.org>
Subject Kafka Mirroring setup
Date Tue, 20 Aug 2013 17:35:13 GMT
Hi all!

Wikimedia is investigating how best to set up Broker clusters in multiple data centers.  Our
main analytics Broker cluster is currently in our main datacenter.  It is possible for all
of the main DC's frontend producers to produce directly to our analytics cluster, but we're
not sure if this is a best practice.  So!  What does LinkedIn recommend?

Option A: N + 1 clusters.
- N production Broker Clusters (1 for each DC).
- +1 aggregator/analytics Broker cluster that mirrors all of the production clusters.

- Option B: N total Broker clusters.
- Frontend producers in the main cluster produce directly to the aggregator/analytics cluster.
- Other DC's clusters are mirrored to the aggregator/analytics cluster.

View raw message