kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From François Méthot <fmetho...@gmail.com>
Subject Number of Partitions and Performance
Date Tue, 07 Apr 2015 17:13:27 GMT
Hi,

  We initially had configured our topics to have between 8 to 16 partitions
each on a cluster of 10 brokers (vm with 2 cores, 16 MB ram, Few TB of SAN
Disk).

Then I came across the rule of thump formula *100 x b x r.*
(
http://blog.confluent.io/2015/03/12/how-to-choose-the-number-of-topicspartitions-in-a-kafka-cluster/
)

100 x 10 brokers x 2 Replication = 2000 partitions.

We gave it try and but our single threaded kafka producer performance
dropped by 80%.

What is the benefits of having that much partitions?

Is there any problem in the long run with using a topic with as few as 16
partitions?


Francois

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message