kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Lloyd <gllo...@gmail.com>
Subject Re: consumer groups
Date Wed, 17 Dec 2014 22:33:18 GMT
Thanks for the reply,

So if I wanted to add a new group of consumers 6 months into the lifespan
of my implementation and I didn't want that new group to process all the
last six months is there a method to manage this?



On Tue, Dec 16, 2014 at 9:48 PM, Gwen Shapira <gshapira@cloudera.com> wrote:
>
> " If all
> the consumers stop listening how long will Kafka continue to store messages
> for that group?"
>
> Kafka retains data for set amount of time, regardless of whether
> anyone is listening or not. This amount of time is configurable.
> Because Kafka performance is generally constant with the amount of
> data it stores, storing terabytes of data for months is not uncommon.
>
> Gwen
>
> On Tue, Dec 16, 2014 at 1:09 PM, Greg Lloyd <glloyd1@gmail.com> wrote:
> > Hi,
> >
> > I am planning to use kafka for a work queue type use case with multiple
> > consumers. I also plan to use it with multiple consumer groups. What is
> not
> > clear to me from the documentation is how to define+manage consumer
> groups
> > or if that is possible at all. It appears to me that a consumer group is
> > created just by connecting with a new group-id. What is not clear to me
> is
> > how to manage this: if I no longer want a consumer group how do I remove
> > it? What happens when there are no consumers of a group listening? If all
> > the consumers stop listening how long will Kafka continue to store
> messages
> > for that group?
> >
> > Thanks for any clarification,
> >
> > Greg Lloyd
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message