kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: Distribution of messages among consumers in a consumer group.
Date Tue, 11 Dec 2012 17:32:04 GMT
Currently, partition is the smallest unit that we distribute data among
consumers (in the same consumer group). So, if the # of consumers is larger
than the total number of partitions in a Kafka cluster (across all
brokers), some consumers will never get any data. The solution is to
increase the number of partitions on the broker.

Thanks,

Jun

On Tue, Dec 11, 2012 at 2:19 AM, Jain Rahul <jainr@ivycomptech.com> wrote:

> Hi,
>
> I am trying kafka for one of POC. It seems to working well, but I'm having
> a small doubt about distribution of messages among consumers in same
> consumer group.
>
> Below is my setup details.
>
>
> 1.       Kafka 0.7.1,
>
> 2.       1 producer, 1 broker, 2 Consumer(Separate JVM process on same
> machine) in same consumer group(groupid), 1 topic.
>
> 3.       No partitions set, producer and consumer both connects to
> zookeeper
>
> It does seems that at a time only one consumer seems to getting all
> messages, while the other consumer is sitting ideal.  Is it expected
> behavior or am I missing something.
>
> As per documents, it seems to me same like jms queue system. Can someone
> please guide me.
>
> Regards,
> Rahul
> This email and any attachments are confidential, and may be legally
> privileged and protected by copyright. If you are not the intended
> recipient dissemination or copying of this email is prohibited. If you have
> received this in error, please notify the sender by replying by email and
> then delete the email completely from your system. Any views or opinions
> are solely those of the sender. This communication is not intended to form
> a binding contract unless expressly indicated to the contrary and properly
> authorised. Any actions taken on the basis of this email are at the
> recipient's own risk.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message