kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 永辉 赵 <zhaoyong...@gmail.com>
Subject Re: How does a new broker decide the partitions of the topic?
Date Fri, 28 Dec 2012 04:08:48 GMT
Here is my understanding, but may be wrong.

Every topic in one broker has num.partitions partitions if we only set
num.partitions

When a new broker is added, if there is topic data folder under kafka-logs
folder, 
then the broker will register num.partitions of this topic in zk, else the
broker doesn't know which topic should be registered.
If no brokers register the topic(topic data folder is not existed in any
broker), consumer will fall in bootstrap, assuming each broker has 1
partition.
The broker will register num.partitions of this topic in zk, after it
receive this topic message .

If only a few brokers haven't register the topic in zk, I think these
brokers may not actually work for this topic.

Please correct me if I am wrong.

Thanks,
Yonghui 





On 12-12-28 上午10:30, "王国栋" <wanggd04@gmail.com> wrote:

>Hi ,
>
>I am reading the design doc for Kafka. But  I can not find the algorithm
>which illustrates such scenario. When a new broker is registered in
>zookeeper, how do all the brokers negotiates with all the others? How does
>a new broker decide which partitions should be stored ?
>
>By the way, I find num.partitions in server configure, but I can not find
>any parameter defining the topics of this broker.
>
>Could you tell me how to configure brokers in the following example. For
>example, I want to set "broker1" to store "topic1" and "topic2", each
>topic
>has 3 partitions?
>I think setting topic.partition.count.map as "topic1:3, topic2:3" should
>works. Is there any simpler configuration?
>
>
>Thanks.
>
>-- 
>Guodong Wang(王国栋)
>Email:wanggd04@gmail.com



Mime
View raw message