kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damian Guy <damian....@gmail.com>
Subject Re: checking consumer lag on KStreams app?
Date Tue, 13 Dec 2016 10:13:43 GMT
That PR didn't resolve the issue - see the comment. It was moved to:
https://issues.apache.org/jira/browse/KAFKA-3853
Which has not been merged

On Tue, 13 Dec 2016 at 10:08 Sam Pegler <sam.pegler@infectiousmedia.com>
wrote:

> Is the following PR present https://github.com/apache/kafka/pull/1336?
>
> __
>
> Sam Pegler
>
> PRODUCTION ENGINEER
>
> T. +44(0) 07 562 867 486
>
> <http://www.infectiousmedia.com/>
> 3-7 Herbal Hill / London / EC1R 5EJ
> www.infectiousmedia.com
>
> This email and any attachments are confidential and may also be privileged.
> If you
> are not the intended recipient, please notify the sender immediately, and
> do not
> disclose the contents to another person, use it for any purpose, or store,
> or copy
> the information in any medium. Please also destroy and delete the message
> from
> your computer.
>
>
> On 13 December 2016 at 10:03, Sachin Mittal <sjmittal@gmail.com> wrote:
>
> > If this is a bug then it is not fixed because I just build kafka from
> > source and it gave me the reported error.
> >
> > On Tue, Dec 13, 2016 at 3:24 PM, Sam Pegler <sam.pegler@infectiousmedia.
> > com>
> > wrote:
> >
> > > >You can only check the offsets when there is an active member of the
> > > consumer group.
> > >
> > > This was a bug [1] thats been fixed.  Thanks to Vincent Dautremont for
> > > pointing this out to me a while ago.
> > >
> > > http://mail-archives.apache.org/mod_mbox/kafka-users/201611.mbox/%
> > > 3CCAD2WViSAgwc9i4-9xEw1oz1xzpsbveFt1%3DSZ0qkHRiFEc3fXbw%40mail.
> > > gmail.com%3E
> > >
> > > __
> > >
> > > Sam Pegler
> > >
> > > PRODUCTION ENGINEER
> > >
> > > T. +44(0) 07 562 867 486
> > >
> > > <http://www.infectiousmedia.com/>
> > > 3-7 Herbal Hill / London / EC1R 5EJ
> > > www.infectiousmedia.com
> > >
> > > This email and any attachments are confidential and may also be
> > privileged.
> > > If you
> > > are not the intended recipient, please notify the sender immediately,
> and
> > > do not
> > > disclose the contents to another person, use it for any purpose, or
> > store,
> > > or copy
> > > the information in any medium. Please also destroy and delete the
> message
> > > from
> > > your computer.
> > >
> > >
> > > On 13 December 2016 at 09:39, Damian Guy <damian.guy@gmail.com> wrote:
> > >
> > > > Hi Sachin
> > > >
> > > > That is correct. You can only check the offsets when there is an
> active
> > > > member of the consumer group. In this case that would mean that you
> > have
> > > at
> > > > least one instance of your streams application running.
> > > >
> > > > Thanks,
> > > > Damian
> > > >
> > > > On Tue, 13 Dec 2016 at 06:58 Sachin Mittal <sjmittal@gmail.com>
> wrote:
> > > >
> > > > > Hi,
> > > > > I used the following command
> > > > > bin\windows\kafka-consumer-groups.bat --bootstrap-server
> > > localhost:9092
> > > > > --describe --group test
> > > > > and I get the following output
> > > > >
> > > > > Note: This will only show information about consumers that use the
> > Java
> > > > > consumer API (non-ZooKeeper-based consumers).
> > > > >
> > > > > Error: Consumer group 'test' has no active members.
> > > > >
> > > > > What does this mean.
> > > > >
> > > > > It means I can check the offset of consumer only when streams
> > > applictaion
> > > > > "test" is running.
> > > > >
> > > > > Thanks
> > > > > Sachin
> > > > >
> > > > >
> > > > > On Mon, Dec 12, 2016 at 8:33 PM, Damian Guy <damian.guy@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Hi Sachin,
> > > > > >
> > > > > > You should use the kafka-consumer-groups.sh command. The
> > > > > > ConsumerOffsetChecker is deprecated and is only for the old
> > consumer.
> > > > > >
> > > > > > Thanks,
> > > > > > Damian
> > > > > >
> > > > > > On Mon, 12 Dec 2016 at 14:32 Sachin Mittal <sjmittal@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > > I have a streams application running with application id
test.
> > > > > > > When I try to check consumer lag like you suggested I get
the
> > > > following
> > > > > > > issue:
> > > > > > >
> > > > > > > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker
> > > --zookeeper
> > > > > > > localhost:2181 --group test
> > > > > > > [2016-12-12 10:26:01,348] WARN WARNING: ConsumerOffsetChecker
> is
> > > > > > > deprecated and will be dropped in releases following 0.9.0.
Use
> > > > > > > ConsumerGroupCommand instead. (kafka.tools.
> > ConsumerOffsetChecker$)
> > > > > > > SLF4J: Class path contains multiple SLF4J bindings.
> > > > > > > SLF4J: Found binding in
> > > > > > >
> > > > > > > [jar:file:/home/testuser/kafka/kafka_2.10-0.10.0.1/
> > > > > > libs/logback-classic-1.0.3.jar!/org/slf4j/impl/
> > > > StaticLoggerBinder.class]
> > > > > > > SLF4J: Found binding in
> > > > > > >
> > > > > > > [jar:file:/home/testuser/kafka/kafka_2.10-0.10.0.1/
> > > > > > libs/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/
> > > > StaticLoggerBinder.class]
> > > > > > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
> for
> > > an
> > > > > > > explanation.
> > > > > > > SLF4J: Actual binding is of type
> > > > > > > [ch.qos.logback.classic.selector.DefaultContextSelector]
> > > > > > > Exiting due to: org.apache.zookeeper.KeeperException$
> > > > NoNodeException:
> > > > > > > KeeperErrorCode = NoNode for /consumers/test/owners.
> > > > > > >
> > > > > > > Please let me know where I may be going wrong.
> > > > > > > I have the kafka logs set in folder
> > > > > > > /data01/testuser/kafka-logs
> > > > > > >
> > > > > > > Under kafka-logs I see many folders with name something
like
> > > > > > > consumer_offsets_*
> > > > > > >
> > > > > > > I have the stream dir set in folder
> > > > > > > /data01/testuser/kafka-streams/test
> > > > > > >
> > > > > > > Thanks
> > > > > > > Sachin
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Dec 11, 2016 at 2:19 AM, Matthias J. Sax <
> > > > > matthias@confluent.io>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > It's basically just a consumer as any other. The
> > application.id
> > > is
> > > > > > used
> > > > > > > > as consumer group.id.
> > > > > > > >
> > > > > > > > So just use the available tools you do use to check
consumer
> > lag.
> > > > > > > >
> > > > > > > >
> > > > > > > > -Matthias
> > > > > > > >
> > > > > > > > On 12/9/16 5:49 PM, Jon Yeargers wrote:
> > > > > > > > > How would this be done?
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message