kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From François Méthot <fmetho...@gmail.com>
Subject Number of Partitions and Performance
Date Tue, 07 Apr 2015 17:13:27 GMT

  We initially had configured our topics to have between 8 to 16 partitions
each on a cluster of 10 brokers (vm with 2 cores, 16 MB ram, Few TB of SAN

Then I came across the rule of thump formula *100 x b x r.*

100 x 10 brokers x 2 Replication = 2000 partitions.

We gave it try and but our single threaded kafka producer performance
dropped by 80%.

What is the benefits of having that much partitions?

Is there any problem in the long run with using a topic with as few as 16


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message