kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Neha Narkhede <neha.narkh...@gmail.com>
Subject Re: Consumer group ID for high level consumer
Date Mon, 03 Mar 2014 17:25:22 GMT
Question  : How to make the consumer threads receive the older data that
was produced when the consumer threads aren't running ?

As long as you make sure the group.id doesn't change, the consumer will
always
consume data from where it left off during its last run. In other words,
the consumer
can consume data that was produced when it wasn't running, if it is always
started
using the same group.id

Thanks,
Neha


On Mon, Mar 3, 2014 at 8:02 AM, Binita Bharati <binita.bharati@gmail.com>wrote:

> Hi all,
>
> The HighLevelConsumer is working fine. Ref :
> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example.
>
> I was expecting the consumer threads to be able to receive older data ( ie
> the data put into the topic when the consumer threads weren't running )
> That was the root of the problem, and once, I started my consumer
> application first, and then, generated fresh data into the topic, the
> consumer threads started receiving the new data.
>
> Question  : How to make the consumer threads receive the older data that
> was produced when the consumer threads aren't running ?
>
> Thanks
>
>
> On Wed, Feb 26, 2014 at 6:30 PM, Martin Kleppmann
> <mkleppmann@linkedin.com>wrote:
>
> > Hi Binita,
> >
> > The consumer group (group.id) is a mechanism for sharing the load of
> > consuming a high-volume topic between multiple consumers. If you don't
> set
> > a group ID, each consumer consumes all the partitions of a topic. If you
> > set several consumers to the same group ID, the partitions of the topic
> > you're consuming will be shared amongst the consumers in that group, so
> > that each message is delivered to only one of the consumers in the group.
> >
> > You can set the group ID to be whatever you like.
> >
> > Hope that helps,
> > Martin
> >
> > On 25 Feb 2014, at 06:27, Binita Bharati <binita.bharati@gmail.com>
> wrote:
> > > Hi all,
> > >
> > > I am referring to this e.g:
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> > .
> > >
> > > What is the consumer group ID being referred here ?
> > >
> > > Thanks
> > > Binita
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message