kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 永辉 赵 <zhaoyong...@gmail.com>
Subject Re: How does a new broker decide the partitions of the topic?
Date Sun, 30 Dec 2012 17:45:51 GMT
Partition name registered in zk contains broker id and sequence(partition)
id, such as "0-0", "0-1", ...

Thanks,
Yonghui 





On 12-12-30 下午11:44, "王国栋" <wanggd04@gmail.com> wrote:

>Thanks Yonghui.
>
>But how does the new broker decide the partition id of the new topic? Or
>each partion id contains a broker id and a sequence id?
>
>
>
>
>On Fri, Dec 28, 2012 at 12:08 PM, 永辉 赵 <zhaoyonghui@gmail.com> wrote:
>
>> Here is my understanding, but may be wrong.
>>
>> Every topic in one broker has num.partitions partitions if we only set
>> num.partitions
>>
>> When a new broker is added, if there is topic data folder under
>>kafka-logs
>> folder,
>> then the broker will register num.partitions of this topic in zk, else
>>the
>> broker doesn't know which topic should be registered.
>> If no brokers register the topic(topic data folder is not existed in any
>> broker), consumer will fall in bootstrap, assuming each broker has 1
>> partition.
>> The broker will register num.partitions of this topic in zk, after it
>> receive this topic message .
>>
>> If only a few brokers haven't register the topic in zk, I think these
>> brokers may not actually work for this topic.
>>
>> Please correct me if I am wrong.
>>
>> Thanks,
>> Yonghui
>>
>>
>>
>>
>>
>> On 12-12-28 上午10:30, "王国栋" <wanggd04@gmail.com> wrote:
>>
>> >Hi ,
>> >
>> >I am reading the design doc for Kafka. But  I can not find the
>>algorithm
>> >which illustrates such scenario. When a new broker is registered in
>> >zookeeper, how do all the brokers negotiates with all the others? How
>>does
>> >a new broker decide which partitions should be stored ?
>> >
>> >By the way, I find num.partitions in server configure, but I can not
>>find
>> >any parameter defining the topics of this broker.
>> >
>> >Could you tell me how to configure brokers in the following example.
>>For
>> >example, I want to set "broker1" to store "topic1" and "topic2", each
>> >topic
>> >has 3 partitions?
>> >I think setting topic.partition.count.map as "topic1:3, topic2:3"
>>should
>> >works. Is there any simpler configuration?
>> >
>> >
>> >Thanks.
>> >
>> >--
>> >Guodong Wang(王国栋)
>> >Email:wanggd04@gmail.com
>>
>>
>>
>
>
>-- 
>Guodong Wang
>王国栋



Mime
View raw message