kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Binita Bharati <binita.bhar...@gmail.com>
Subject Re: Consumer group ID for high level consumer
Date Mon, 03 Mar 2014 16:02:56 GMT
Hi all,

The HighLevelConsumer is working fine. Ref :
https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example.

I was expecting the consumer threads to be able to receive older data ( ie
the data put into the topic when the consumer threads weren't running )
That was the root of the problem, and once, I started my consumer
application first, and then, generated fresh data into the topic, the
consumer threads started receiving the new data.

Question  : How to make the consumer threads receive the older data that
was produced when the consumer threads aren't running ?

Thanks


On Wed, Feb 26, 2014 at 6:30 PM, Martin Kleppmann
<mkleppmann@linkedin.com>wrote:

> Hi Binita,
>
> The consumer group (group.id) is a mechanism for sharing the load of
> consuming a high-volume topic between multiple consumers. If you don't set
> a group ID, each consumer consumes all the partitions of a topic. If you
> set several consumers to the same group ID, the partitions of the topic
> you're consuming will be shared amongst the consumers in that group, so
> that each message is delivered to only one of the consumers in the group.
>
> You can set the group ID to be whatever you like.
>
> Hope that helps,
> Martin
>
> On 25 Feb 2014, at 06:27, Binita Bharati <binita.bharati@gmail.com> wrote:
> > Hi all,
> >
> > I am referring to this e.g:
> > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> .
> >
> > What is the consumer group ID being referred here ?
> >
> > Thanks
> > Binita
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message