kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pavel Sýkora <pav...@seznam.cz>
Subject Empty Partitions Impact on Performance
Date Wed, 06 May 2015 11:32:49 GMT
Hi,



I've started playing with Kafka_2.10-0.8.2.1 (a cluster of 3 VM nodes). I 
found a strange behaviour: Inbound throughput was about 60 thousand messages
per second for a partition with 50 partitions. But after adding topics with 
950 partitions total, the performance dropped under 25 thousand messages per
second, even if the 950 new partitions were unused and empty. In other 
words: a simple existence of empty unused partitions can drop Kafka 
performance to less than 50 %.




Has anybody else observed such Kafka behaviour? Does anybody have an 
explanation for this?




Regards, 




Pavel 
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message