kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yury Ruchin <yuri.ruc...@gmail.com>
Subject Re: Kafka 0.8's VerifyConsumerRebalance reports an error
Date Tue, 24 Jun 2014 18:28:19 GMT
I've just double-checked. The URL is correct, the same one is used by Kafka
clients.


2014-06-24 22:21 GMT+04:00 Neha Narkhede <neha.narkhede@gmail.com>:

> Is it possible that maybe the zookeeper url used for the
> VerifyConsumerRebalance tool is incorrect?
>
>
> On Tue, Jun 24, 2014 at 12:02 AM, Yury Ruchin <yuri.ruchin@gmail.com>
> wrote:
>
> > Hi,
> >
> > I've run into the following problem. I try to read from a 50-partition
> > Kafka topic using high level consumer with 8 streams. I'm using 8-thread
> > pool, each thread handling one stream. After a short time, the threads
> > reading from the stream stop reading. Lag between topic latest offset and
> > the consumer constantly increases as new messages come in.
> >
> > I looked into ZK:  /consumers/<consumer_name>/owners/<topic_name> and
> see a
> > list of znodes corresponding to the full list of partitions: [1, 2, 3,
> > ...]. When I do zk get on e. g.
> > /consumers/<consumer_name>/owners/<topic_name>/1 - I see a valid consumer
> > name corresponding to Kafka client logs, e. g.
> > <consumer_name>_<node_name>-1403049600000-abc12345-0. However, when
I run
> > the VerifyConsumerRebalance tool, I see the following:
> >
> > No owner for partition [<topic_name>,1]
> > (kafka.tools.VerifyConsumerRebalance$)
> >
> > No owner for partition [<topic_name>,2]
> > (kafka.tools.VerifyConsumerRebalance$)
> >
> > ...
> >
> > No owner for partition [<topic_name>,50]
> > (kafka.tools.VerifyConsumerRebalance$)
> >
> > According to this output, no partition has owner, which seemingly
> > contradicts to what I see in ZK.
> >
> > What would cause such a problem and how can I troubleshoot it further?
> >
> > Thanks!
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message