kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Distribution of messages among consumers in a consumer group.
Date Tue, 11 Dec 2012 17:32:04 GMT
Currently, partition is the smallest unit that we distribute data among
consumers (in the same consumer group). So, if the # of consumers is larger
than the total number of partitions in a Kafka cluster (across all
brokers), some consumers will never get any data. The solution is to
increase the number of partitions on the broker.



On Tue, Dec 11, 2012 at 2:19 AM, Jain Rahul <jainr@ivycomptech.com> wrote:

> Hi,
> I am trying kafka for one of POC. It seems to working well, but I'm having
> a small doubt about distribution of messages among consumers in same
> consumer group.
> Below is my setup details.
> 1.       Kafka 0.7.1,
> 2.       1 producer, 1 broker, 2 Consumer(Separate JVM process on same
> machine) in same consumer group(groupid), 1 topic.
> 3.       No partitions set, producer and consumer both connects to
> zookeeper
> It does seems that at a time only one consumer seems to getting all
> messages, while the other consumer is sitting ideal.  Is it expected
> behavior or am I missing something.
> As per documents, it seems to me same like jms queue system. Can someone
> please guide me.
> Regards,
> Rahul
> This email and any attachments are confidential, and may be legally
> privileged and protected by copyright. If you are not the intended
> recipient dissemination or copying of this email is prohibited. If you have
> received this in error, please notify the sender by replying by email and
> then delete the email completely from your system. Any views or opinions
> are solely those of the sender. This communication is not intended to form
> a binding contract unless expressly indicated to the contrary and properly
> authorised. Any actions taken on the basis of this email are at the
> recipient's own risk.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message