kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Corley <davidcor...@gmail.com>
Subject Re: Horizontally Scaling Kafka Consumers
Date Wed, 29 Apr 2015 12:25:19 GMT
If the 100 partitions are all for the same topic, you can have up to 100
consumers working as part of a single consumer group for that topic.
You cannot have more consumers than there are partitions within a given
consumer group.

On 29 April 2015 at 08:41, Nimi Wariboko Jr <nimi@channelmeter.com> wrote:

> Hi,
>
> I was wondering what options there are for horizontally scaling kafka
> consumers? Basically if I have 100 partitions and 10 consumers, and want to
> temporarily scale up to 50 consumers, what options do I have?
>
> So far I've thought of just simply tracking consumer membership somehow
> (either through Raft or zookeeper's znodes) on the consumers.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message