kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: multi-broker topic rebalance
Date Tue, 27 May 2014 03:28:45 GMT
Which version of Kafka are you using? How many partitions do you have in
that topic?

Thanks,

Jun


On Mon, May 26, 2014 at 7:14 AM, Рябков Алексей Николаевич <
a.ryabkov@ntc-vulkan.ru> wrote:

>
> Hello!
>
> How can I rebalance topic within multiple brokers?
> For example:
>
> 1.        I have about 30k consumers...
>
> 2.        I need to use 3 broker (10k consumer per broker)
>
> 3.       I use multiple (around 10) producers with default balance strategy
> Sometimes one broker have empty topic (consumer  finished all message on
> broker)...But another broker have a lot of messages in topic...
>
> How to resolve such situation:
> -i cant just switch my consumers to another broker ( broker can not handle
> more than 10k consumers....)
> -I can not find how to rebalance topic within multiple brokers...
> - I can not find how to setup limit for message per topic (or partition)
> ... If such things exists I can handle such case when add new message to
> broker in consumer side...and switch to  another ...
>
> Thanks in advance, Aleksey Ryabkov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message