kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Becket Qin <becket....@gmail.com>
Subject Re: Checking the consumer lag when using manual partition assignment with the KafkaConsumer
Date Mon, 07 Nov 2016 01:39:07 GMT
Hi Robert,

Have you tried Kafka-consumer-offset-checker.sh come with Kafka? It should
work regardless whether Kafka based group management is used or not.

Thanks,

Jiangjie (Becket) Qin

On Sun, Nov 6, 2016 at 4:00 PM, Vincent Dautremont <
vincent.dautremont@olamobile.com> wrote:

> By the way I remember having read somewhere on this list that this utility
> not showing info for consumer groups that do not have current active
> consumers was a bug .
>
> That would be a thing to fix, is there an expected fix date / fix release
> for this?
>
>
> > Le 6 nov. 2016 à 13:23, Robert Metzger <rmetzger@apache.org> a écrit :
> >
> > Hi Matthias,
> >
> > the bin/kafka-consumer-groups.sh utility is exactly what the user in
> > http://grokbase.com/t/kafka/users/163rrq9ne8/new-consumer-
> group-not-showing-up
> > has used.
> > It seems that the broker is not returning consumer groups that don't
> > participate in the consumer group balancing mechanism.
> >
> >
> > On Fri, Nov 4, 2016 at 6:07 PM, Matthias J. Sax <matthias@confluent.io>
> > wrote:
> >
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA512
> >>
> >> Robert,
> >>
> >> you can use bin/kafka-consumer-groups.sh instead.
> >>
> >>> bin/kafka-consumer-offset-checker.sh [2016-11-04 10:05:07,852] WARN
> >>> WARNING: ConsumerOffsetChecker is deprecated and will be dropped in
> >>> releases following 0.9.0. Use ConsumerGroupCommand instead.
> >>> (kafka.tools.ConsumerOffsetChecker$)
> >>
> >>
> >> - -Matthias
> >>
> >>> On 11/3/16 4:07 AM, Robert Metzger wrote:
> >>> Hi,
> >>>
> >>> some Flink users recently noticed that they can not check the
> >>> consumer lag when using Flink's kafka consumer [1]. According to
> >>> this discussion on the Kafka user list [2] the
> >>> kafka-consumer-groups.sh utility doesn't work with KafkaConsumers
> >>> with manual partition assignment.
> >>>
> >>> Is there a way to get the consumer lag for a consumer that
> >>> regularly commits its offsets to the broker?
> >>>
> >>> Regards, Robert
> >>>
> >>>
> >>> [1] https://issues.apache.org/jira/browse/FLINK-5001 [2]
> >>> http://grokbase.com/t/kafka/users/163rrq9ne8/new-consumer-
> >> group-not-showing-up
> >>>
> >> -----BEGIN PGP SIGNATURE-----
> >> Comment: GPGTools - https://gpgtools.org
> >>
> >> iQIcBAEBCgAGBQJYHMBkAAoJECnhiMLycopPJvEQALdx2dOVa30QsDBtNIJAzwwJ
> >> tk+ZS3lgefNkAWYBMccNTDFqDH0OtLKUhi9Pproye0LTItlaNGYgemC7kzzPnlDv
> >> R6PrJe+GzRNd13eNSBzb7kJ9nxodsZ9flpUsMrX3k2g3v6w6cwV7Xon1LlHV5HtO
> >> xbJ0ei41lNcBFjbgYFbwyzsS+W3FQt1FTw/we4mL3TwaBYwHG5pfByK0eC0NUEEy
> >> Q2rSPnZ8701igC4RJoWRIe0QibzZSdzEk/S8rWpNxTrmG/TaBhM2mObbf96iqM0J
> >> VvJcUiuUpAY/CKiOY6tjCGRpCAGYdrdPismaTQA6hGxzr+91l3eXebn1lFEw2Auc
> >> E0fbkZ5XoNo/JQaHcDA2M+iwVZZMIOP/yfierkG7g3XmYfbCpIKd9+AU79GYGwe+
> >> nhCUB2BbhEi+saNharAHtm6ai1T55bmSnCBYiz14wJpEYYzDJv1BnRN9YvV3Topl
> >> GNMSK3/zWIALXPp8fFE0JFxX0syaGRqMS5XqlsnklulCaeB/M1gWwGyKxYF+cAWG
> >> 5F5kuKGvr8YgNmRPSOAO8sastbL9se1Kw6uDQj0eN8kimbXptCw+LtEOlQ3z9cgb
> >> IQcbl33oJUcUJdUejQ34rGK84P59CVzIt2R7r722QwJYE9pWBIR9S/+Ft4QnkBaE
> >> VnmHxH/PmQrXMTl8DYs1
> >> =wVbw
> >> -----END PGP SIGNATURE-----
> >>
>
> --
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and/or privileged
> material. Any review, retransmission, dissemination or other use of, or
> taking of any action in reliance upon, this information by persons or
> entities other than the intended recipient is prohibited. If you received
> this in error, please contact the sender and delete the material from any
> computer.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message