kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From S L <slouie.at.w...@gmail.com>
Subject Re: Is there a way to get the offset of a consumer of a topic?
Date Fri, 06 Dec 2013 22:11:49 GMT
thank you all for your help and suggestion.  The ConsumerOffsetChecker is
exactly what I needed!  thanks again!


On Wed, Dec 4, 2013 at 1:03 PM, Philip O'Toole <philip@loggly.com> wrote:

> Simple tool I wrote to monitor 0.7 consumers.
>
> https://github.com/otoolep/stormkafkamon
>
>
> On Wed, Dec 4, 2013 at 12:49 PM, David DeMaagd <ddemaagd@linkedin.com
> >wrote:
>
> > You can use either the MaxLag MBean (0.8):
> >
> > http://kafka.apache.org/documentation.html#monitoring
> >
> > Or the ConsumerOffsetChecker (0.7 or 0.8, can't seem to find a doc
> > reference for it):
> >
> > ./kafka-run-class.sh kafka.tools.ConsumerOffsetChecker ...
> >
> > --
> > Dave DeMaagd | S'aite Reliability Engineering, Y'all
> > ddemaagd@linkedin.com | 818 262 7958
> >
> > (slouie.at.work@gmail.com - Wed, Dec 04, 2013 at 12:40:59PM -0800)
> > > Hi,
> > > I was wondering if there's an easy way to get the current offset of a
> > > specified consumer?  What we need is something to check the consumer
> > offset
> > > every 5 min.  If it's changing at a certain rate every time we check,
> > good.
> > >  If not, then we have a problem.  I was going to write a monitoring
> > program
> > > to check the broker or consumer for this offset to see if it changed
> and
> > > how much it changed since the last time I checked it.
> > >
> > > I ran into this pg on the wiki which looks like it's what I need.
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommit%2FFetchAPI
> > >
> > > However, it says the API calls are not functional yet.  Are there some
> > > other methods or APIs or JMX bean/object/etc that we can use in the
> > > meantime?  If I were to ping zookeeper, where do I look for this
> offset?
> > >  Thanks in advance for your help.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message