kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vahid S Hashemian" <vahidhashem...@us.ibm.com>
Subject Re: Kafka 0.10.1.0 - Question about kafka-consumer-groups.sh
Date Fri, 05 May 2017 21:46:25 GMT
Hi Subhash,

Yes, it should be listed again in the output, but it should get fresh 
offsets (with `--describe` for example), since the old offsets were 
removed once it became inactive.

--Vahid




From:   Subhash Sriram <subhash.sriram@gmail.com>
To:     users@kafka.apache.org
Date:   05/05/2017 02:38 PM
Subject:        Re: Kafka 0.10.1.0 - Question about 
kafka-consumer-groups.sh



Hi Vahid,

Thank you very much for your reply! I appreciate the clarification.
Unfortunately, I didn't really try the command until today. That being
said, I created a couple of new groups and consumed from a test topic
today, and they did show up in the list. Maybe I can see what happens with
them after the offset.retention.minutes has elapsed.

What if a group went inactive, and then became active again with the same
group name? I would assume that once it turns back on, it would show up in
the list.

Thanks again,
Subhash


On Fri, May 5, 2017 at 5:30 PM, Vahid S Hashemian 
<vahidhashemian@us.ibm.com
> wrote:

> Hi Subhash,
>
> The broker config that affects group offset retention is
> "offsets.retention.minutes" (which defaults to 1 day).
> If the group is inactive (i.e. has no consumer consuming from it) for 
this
> long its offsets will be removed from the internal topic offset and it
> will not be listed in the consumer group command output.
>
> But the consumer group in your case should be alive, since it did not
> become inactive.
>
> Did the command use to list the group in the output before?
>
> --Vahid
>
>
>
>
> From:   Subhash Sriram <subhash.sriram@gmail.com>
> To:     users@kafka.apache.org
> Date:   05/05/2017 01:43 PM
> Subject:        Kafka 0.10.1.0 - Question about kafka-consumer-groups.sh
>
>
>
> Hey everyone,
>
> I am a little bit confused about how the kafka-consumer-groups.sh/
> ConsumerGroupCommand works, and was hoping someone could shed some light
> on
> this for me.
>
> We are running Kafka 0.10.1.0, and using the new Consumer API with the
> Confluent.Kafka C# library (v0.9.5) that uses librdkafka. Today, I was
> trying to get some details on what consumers were running, and their
> position within a couple of topics, but when I ran the following, I did
> not
> see the group in the list.
>
> ./bin/kafka-consumer-groups.sh --bootstrap-server [our servers]
> --new-consumer --list
>
> I see a few groups listed, but none of the ones I was expecting to see. 
I
> saw this in the documentation:
>
> *"When using the new consumer API
> <http://kafka.apache.org/documentation.html#newconsumerapi> (where the
> broker handles coordination of partition handling and rebalance), the
> group
> is deleted when the last committed offset for that group expires"*
>
> Is that related to log retention time? If so, is it saying that the 
group
> will be deleted from the list once the highest committed offset of its
> group is past its configured log retention time?
>
> The issue I am facing is that we have a consumer group that is actively
> consuming. At one point, I am sure the messages in the topic it is
> consuming expired, but since then, more messages have been added, and it
> has been consuming & committing higher offsets. Shouldn't that group 
have
> come back on the list?
>
> Any ideas would be very helpful.
>
> Thanks,
> Subhash
>
>
>
>
>





Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message