kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 4mayank <4may...@gmail.com>
Subject Re: Consumer offsets in offsets topic 0.8.2
Date Tue, 14 Apr 2015 19:03:07 GMT
Thanks guys. Based on the responses it looks like I need to have a tool of
my own to read offsets.

On Mon, Apr 13, 2015 at 3:07 PM, Jiangjie Qin <jqin@linkedin.com.invalid>
wrote:

> Yeah, the current ConsumerOffsetChecker has this issue (maybe bug also) if
> the offset storage is Kafka and no offset has been committed. It will
> throw ZK exception, which is very confusing. KAFKA-1951 was opened for
> this but was not checked in.
>
> Thanks.
>
> Jiangjie (Becket) Qin
>
> On 4/13/15, 9:55 AM, "4mayank" <4mayank@gmail.com> wrote:
>
> >I did a similar change - moved from High Level Consumer to Simple
> >Consumer.
> >Howerver kafka-consumer-offset-checker.sh throws an exception. Its
> >searching the zk path /consumers/<group>/ which does not exist on any of
> >my
> >zk nodes.
> >
> >Is there any other tool for getting the offset lag when using Simple
> >Consumer? Or am I using kafka-consumer-offset-checker.sh incorrectly for
> >Simple Consumer?
> >
> >Output:
> >kafka-consumer-offset-checker.sh --zookeeper 192.168.1.201:2181,
> >192.168.1.202:2181,192.168.1.203:2181 --group my-control-group
> >Exiting due to: org.apache.zookeeper.KeeperException$NoNodeException:
> >KeeperErrorCode = NoNode for /consumers/my-control-group/owners.
> >
> >
> >kafka-consumer-offset-checker.sh --zookeeper 192.168.1.201:2181,
> >192.168.1.202:2181,192.168.1.203:2181 --group my-control-group --topic
> >my-control
> >Exiting due to: org.apache.zookeeper.KeeperException$NoNodeException:
> >KeeperErrorCode = NoNode for
> >/consumers/my-control-group/offsets/my-control/1.
> >
> >
> >ZK cli output:
> >
> >./zookeeper-shell.sh 192.168.1.201:2181
> >Connecting to 192.168.1.201:2181
> >Welcome to ZooKeeper!
> >JLine support is disabled
> >
> >WATCHER::
> >
> >WatchedEvent state:SyncConnected type:None path:null
> >ls /config/topics
> >[my-control]
> >ls /consumers
> >[]
> >quit
> >
> >
> >Thanks.
> >Mayank.
> >
> >On Fri, Mar 20, 2015 at 9:54 AM, Jiangjie Qin <jqin@linkedin.com.invalid>
> >wrote:
> >
> >> Hi Vamsi,
> >>
> >> The ConsumerOffsetChecker.scala or kafka-consumer-offset-checker.sh
> >>still
> >> works. You can use them to check the offsets.
> >> If you need to check the offsets programmatically, you can send
> >> OffsetsFetcheRequest to broker using simple consumer. You may refer the
> >> ConsumerOffsetChecker.scala to see how to find correct broker where
> >> corresponding offset manager resides.
> >>
> >> Jiangjie (Becket) Qin
> >>
> >> On 3/19/15, 11:54 PM, "Achanta Vamsi Subhash"
> >><achanta.vamsi@flipkart.com>
> >> wrote:
> >>
> >> >Hi,
> >> >
> >> >We are using 0.8.2.1 currently.
> >> >
> >> >- How to get the consumer offsets from the offsets topic?
> >> >​- Is there any built-in function which I could use? (like in
> >> >AdminUtils.scala)
> >> >- Is it ok to start a simple consumer and read the offsets from the
> >>topic?
> >> >
> >> >​We used to read the offsets from zookeeper previously for the
> >> >HighLevelConsumers. But with the new broker this changed as we are
> >>using
> >> >Kafka topic for offsets.​
> >> >
> >> >--
> >> >Regards
> >> >Vamsi Subhash
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message