kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid S Hashemian" <vahidhashem...@us.ibm.com>
Subject Re: Kafka 0.10.1.0 - Question about kafka-consumer-groups.sh
Date Fri, 05 May 2017 21:30:31 GMT
Hi Subhash,

The broker config that affects group offset retention is 
"offsets.retention.minutes" (which defaults to 1 day).
If the group is inactive (i.e. has no consumer consuming from it) for this 
long its offsets will be removed from the internal topic offset and it 
will not be listed in the consumer group command output.

But the consumer group in your case should be alive, since it did not 
become inactive.

Did the command use to list the group in the output before?

--Vahid




From:   Subhash Sriram <subhash.sriram@gmail.com>
To:     users@kafka.apache.org
Date:   05/05/2017 01:43 PM
Subject:        Kafka 0.10.1.0 - Question about kafka-consumer-groups.sh



Hey everyone,

I am a little bit confused about how the kafka-consumer-groups.sh/
ConsumerGroupCommand works, and was hoping someone could shed some light 
on
this for me.

We are running Kafka 0.10.1.0, and using the new Consumer API with the
Confluent.Kafka C# library (v0.9.5) that uses librdkafka. Today, I was
trying to get some details on what consumers were running, and their
position within a couple of topics, but when I ran the following, I did 
not
see the group in the list.

./bin/kafka-consumer-groups.sh --bootstrap-server [our servers]
--new-consumer --list

I see a few groups listed, but none of the ones I was expecting to see. I
saw this in the documentation:

*"When using the new consumer API
<http://kafka.apache.org/documentation.html#newconsumerapi> (where the
broker handles coordination of partition handling and rebalance), the 
group
is deleted when the last committed offset for that group expires"*

Is that related to log retention time? If so, is it saying that the group
will be deleted from the list once the highest committed offset of its
group is past its configured log retention time?

The issue I am facing is that we have a consumer group that is actively
consuming. At one point, I am sure the messages in the topic it is
consuming expired, but since then, more messages have been added, and it
has been consuming & committing higher offsets. Shouldn't that group have
come back on the list?

Any ideas would be very helpful.

Thanks,
Subhash





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message