kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonghui Zhao <zhaoyong...@gmail.com>
Subject How kafka assign partition to stream
Date Mon, 10 Mar 2014 17:23:25 GMT
In my environment,  I have 2 brokers and only 1 topic,  each brokers has 10
so there are 20 partitions in total.
I have 4 consumers in one consumer group,  each consumer use
createMessageStreams to create 10 streams, 40 streams in total.

Since partition can not be split,  so there are 20 streams are empty

We found 2 machines have 10 valid streams, but the other 2 have no valid
Is it expected?  Maybe each consumer has 5 valid streams is better for load

And I want to know if  one of the machines which have 10 valid streams is
dead,  will the the partitions be balanced to other machine dynamically?
 i.e. some empty stream in other machines will get data now.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message