kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Pegler <sam.peg...@infectiousmedia.com>
Subject Re: checking consumer lag on KStreams app?
Date Tue, 13 Dec 2016 10:08:07 GMT
Is the following PR present https://github.com/apache/kafka/pull/1336?

__

Sam Pegler

PRODUCTION ENGINEER

T. +44(0) 07 562 867 486

<http://www.infectiousmedia.com/>
3-7 Herbal Hill / London / EC1R 5EJ
www.infectiousmedia.com

This email and any attachments are confidential and may also be privileged.
If you
are not the intended recipient, please notify the sender immediately, and
do not
disclose the contents to another person, use it for any purpose, or store,
or copy
the information in any medium. Please also destroy and delete the message
from
your computer.


On 13 December 2016 at 10:03, Sachin Mittal <sjmittal@gmail.com> wrote:

> If this is a bug then it is not fixed because I just build kafka from
> source and it gave me the reported error.
>
> On Tue, Dec 13, 2016 at 3:24 PM, Sam Pegler <sam.pegler@infectiousmedia.
> com>
> wrote:
>
> > >You can only check the offsets when there is an active member of the
> > consumer group.
> >
> > This was a bug [1] thats been fixed.  Thanks to Vincent Dautremont for
> > pointing this out to me a while ago.
> >
> > http://mail-archives.apache.org/mod_mbox/kafka-users/201611.mbox/%
> > 3CCAD2WViSAgwc9i4-9xEw1oz1xzpsbveFt1%3DSZ0qkHRiFEc3fXbw%40mail.
> > gmail.com%3E
> >
> > __
> >
> > Sam Pegler
> >
> > PRODUCTION ENGINEER
> >
> > T. +44(0) 07 562 867 486
> >
> > <http://www.infectiousmedia.com/>
> > 3-7 Herbal Hill / London / EC1R 5EJ
> > www.infectiousmedia.com
> >
> > This email and any attachments are confidential and may also be
> privileged.
> > If you
> > are not the intended recipient, please notify the sender immediately, and
> > do not
> > disclose the contents to another person, use it for any purpose, or
> store,
> > or copy
> > the information in any medium. Please also destroy and delete the message
> > from
> > your computer.
> >
> >
> > On 13 December 2016 at 09:39, Damian Guy <damian.guy@gmail.com> wrote:
> >
> > > Hi Sachin
> > >
> > > That is correct. You can only check the offsets when there is an active
> > > member of the consumer group. In this case that would mean that you
> have
> > at
> > > least one instance of your streams application running.
> > >
> > > Thanks,
> > > Damian
> > >
> > > On Tue, 13 Dec 2016 at 06:58 Sachin Mittal <sjmittal@gmail.com> wrote:
> > >
> > > > Hi,
> > > > I used the following command
> > > > bin\windows\kafka-consumer-groups.bat --bootstrap-server
> > localhost:9092
> > > > --describe --group test
> > > > and I get the following output
> > > >
> > > > Note: This will only show information about consumers that use the
> Java
> > > > consumer API (non-ZooKeeper-based consumers).
> > > >
> > > > Error: Consumer group 'test' has no active members.
> > > >
> > > > What does this mean.
> > > >
> > > > It means I can check the offset of consumer only when streams
> > applictaion
> > > > "test" is running.
> > > >
> > > > Thanks
> > > > Sachin
> > > >
> > > >
> > > > On Mon, Dec 12, 2016 at 8:33 PM, Damian Guy <damian.guy@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Sachin,
> > > > >
> > > > > You should use the kafka-consumer-groups.sh command. The
> > > > > ConsumerOffsetChecker is deprecated and is only for the old
> consumer.
> > > > >
> > > > > Thanks,
> > > > > Damian
> > > > >
> > > > > On Mon, 12 Dec 2016 at 14:32 Sachin Mittal <sjmittal@gmail.com>
> > wrote:
> > > > >
> > > > > > Hi,
> > > > > > I have a streams application running with application id test.
> > > > > > When I try to check consumer lag like you suggested I get the
> > > following
> > > > > > issue:
> > > > > >
> > > > > > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker
> > --zookeeper
> > > > > > localhost:2181 --group test
> > > > > > [2016-12-12 10:26:01,348] WARN WARNING: ConsumerOffsetChecker
is
> > > > > > deprecated and will be dropped in releases following 0.9.0.
Use
> > > > > > ConsumerGroupCommand instead. (kafka.tools.
> ConsumerOffsetChecker$)
> > > > > > SLF4J: Class path contains multiple SLF4J bindings.
> > > > > > SLF4J: Found binding in
> > > > > >
> > > > > > [jar:file:/home/testuser/kafka/kafka_2.10-0.10.0.1/
> > > > > libs/logback-classic-1.0.3.jar!/org/slf4j/impl/
> > > StaticLoggerBinder.class]
> > > > > > SLF4J: Found binding in
> > > > > >
> > > > > > [jar:file:/home/testuser/kafka/kafka_2.10-0.10.0.1/
> > > > > libs/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/
> > > StaticLoggerBinder.class]
> > > > > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
for
> > an
> > > > > > explanation.
> > > > > > SLF4J: Actual binding is of type
> > > > > > [ch.qos.logback.classic.selector.DefaultContextSelector]
> > > > > > Exiting due to: org.apache.zookeeper.KeeperException$
> > > NoNodeException:
> > > > > > KeeperErrorCode = NoNode for /consumers/test/owners.
> > > > > >
> > > > > > Please let me know where I may be going wrong.
> > > > > > I have the kafka logs set in folder
> > > > > > /data01/testuser/kafka-logs
> > > > > >
> > > > > > Under kafka-logs I see many folders with name something like
> > > > > > consumer_offsets_*
> > > > > >
> > > > > > I have the stream dir set in folder
> > > > > > /data01/testuser/kafka-streams/test
> > > > > >
> > > > > > Thanks
> > > > > > Sachin
> > > > > >
> > > > > >
> > > > > > On Sun, Dec 11, 2016 at 2:19 AM, Matthias J. Sax <
> > > > matthias@confluent.io>
> > > > > > wrote:
> > > > > >
> > > > > > > It's basically just a consumer as any other. The
> application.id
> > is
> > > > > used
> > > > > > > as consumer group.id.
> > > > > > >
> > > > > > > So just use the available tools you do use to check consumer
> lag.
> > > > > > >
> > > > > > >
> > > > > > > -Matthias
> > > > > > >
> > > > > > > On 12/9/16 5:49 PM, Jon Yeargers wrote:
> > > > > > > > How would this be done?
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message