kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Subhash Sriram <subhash.sri...@gmail.com>
Subject Re: Kafka - Question about kafka-consumer-groups.sh
Date Fri, 05 May 2017 21:38:27 GMT
Hi Vahid,

Thank you very much for your reply! I appreciate the clarification.
Unfortunately, I didn't really try the command until today. That being
said, I created a couple of new groups and consumed from a test topic
today, and they did show up in the list. Maybe I can see what happens with
them after the offset.retention.minutes has elapsed.

What if a group went inactive, and then became active again with the same
group name? I would assume that once it turns back on, it would show up in
the list.

Thanks again,

On Fri, May 5, 2017 at 5:30 PM, Vahid S Hashemian <vahidhashemian@us.ibm.com
> wrote:

> Hi Subhash,
> The broker config that affects group offset retention is
> "offsets.retention.minutes" (which defaults to 1 day).
> If the group is inactive (i.e. has no consumer consuming from it) for this
> long its offsets will be removed from the internal topic offset and it
> will not be listed in the consumer group command output.
> But the consumer group in your case should be alive, since it did not
> become inactive.
> Did the command use to list the group in the output before?
> --Vahid
> From:   Subhash Sriram <subhash.sriram@gmail.com>
> To:     users@kafka.apache.org
> Date:   05/05/2017 01:43 PM
> Subject:        Kafka - Question about kafka-consumer-groups.sh
> Hey everyone,
> I am a little bit confused about how the kafka-consumer-groups.sh/
> ConsumerGroupCommand works, and was hoping someone could shed some light
> on
> this for me.
> We are running Kafka, and using the new Consumer API with the
> Confluent.Kafka C# library (v0.9.5) that uses librdkafka. Today, I was
> trying to get some details on what consumers were running, and their
> position within a couple of topics, but when I ran the following, I did
> not
> see the group in the list.
> ./bin/kafka-consumer-groups.sh --bootstrap-server [our servers]
> --new-consumer --list
> I see a few groups listed, but none of the ones I was expecting to see. I
> saw this in the documentation:
> *"When using the new consumer API
> <http://kafka.apache.org/documentation.html#newconsumerapi> (where the
> broker handles coordination of partition handling and rebalance), the
> group
> is deleted when the last committed offset for that group expires"*
> Is that related to log retention time? If so, is it saying that the group
> will be deleted from the list once the highest committed offset of its
> group is past its configured log retention time?
> The issue I am facing is that we have a consumer group that is actively
> consuming. At one point, I am sure the messages in the topic it is
> consuming expired, but since then, more messages have been added, and it
> has been consuming & committing higher offsets. Shouldn't that group have
> come back on the list?
> Any ideas would be very helpful.
> Thanks,
> Subhash

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message