kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Arthur <sart...@salesforce.com>
Subject Relative cluster sizes and cluster size limits
Date Fri, 02 Aug 2013 18:24:32 GMT

I have a question about scaling the broker count of a Kafka cluster.  We
have a scenario where we'll have two clusters replicating data into a
third.  We're wondering how we should size that third cluster so that it
can handle the volume of messages from the two source clusters.  Should we
just make the number of brokers match? e.g. five brokers in the two source
clusters, therefore 10 in the destination cluster.  In general, what is the
horizontal scaling model we should use?  Also, is there an upper limit to
the number of brokers you should put in a cluster, after which you get
diminishing returns on throughput?

Scott Arthur

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message