kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayuresh Gharat <gharatmayures...@gmail.com>
Subject Re: Kafka consumer offset checker hangs indefinitely
Date Mon, 11 May 2015 23:34:56 GMT
Hi Meghana,

Let me try this out on my cluster that has latest trunk deployed.

Thanks,

Mayuresh

On Mon, May 11, 2015 at 1:53 PM, Meghana Narasimhan <
mnarasimhan@bandwidth.com> wrote:

> Hi Mayuresh,
> A small update. The Kafka version I'm currently using is  2.10-0.8.2.1 (not
> 2.11 as previously mentioned). The cluster looks fine. Not sure why the
> consumer offset checker does not return a valid output and gets stuck.
>
> bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test
> Topic:test   PartitionCount:3        ReplicationFactor:3
> Configs:min.insync.replicas=2
>         Topic: test  Partition: 0    Leader: 1       Replicas: 1,2,0 Isr:
> 1,2,0
>         Topic: test  Partition: 1    Leader: 2       Replicas: 2,0,1 Isr:
> 1,2,0
>         Topic: test  Partition: 2    Leader: 0       Replicas: 0,1,2 Isr:
> 1,2,0
>
>
>
>
> On Fri, May 8, 2015 at 12:52 PM, Meghana Narasimhan <
> mnarasimhan@bandwidth.com> wrote:
>
> > Hi Mayuresh,
> >
> > Yes, the broker is up and accepting connections. Multiple consumers are
> > consuming off topics on the broker.
> > Also I am seeing the issue only with this particular version (
> > 2.11-0.8.2.1). It worked fine with the beta that I was using earlier.
> >
> >
> > On Fri, May 8, 2015 at 12:45 PM, Mayuresh Gharat <
> > gharatmayuresh15@gmail.com> wrote:
> >
> >> Is X.X.X.X:9092 up and accepting connections?
> >> I am confused aas in why is it not connecting some other broker if
> >> connection to this broker fails. Can you check if the broker is up?
> >>
> >> The way it works is the consumer will send a ConsumerMetadataRequest to
> >> one
> >> of the brokers and get the offsetmanager for its group and then perform
> >> the
> >> offset management.
> >>
> >> Thanks,
> >>
> >> Mayuresh
> >>
> >> On Fri, May 8, 2015 at 9:22 AM, Meghana Narasimhan <
> >> mnarasimhan@bandwidth.com> wrote:
> >>
> >> > Hi,
> >> > I'm using the Kafka 8.2.1 version(kafka_2.11-0.8.2.1) and the consumer
> >> > offset checker hangs indefinitely and does not return any results. I
> >> > enabled the debug for tools and below is the debug statements as seen
> on
> >> > the stdout. Any thoughts or inputs on this will be much appreciated.
> >> >
> >> > command used :
> >> > bin/kafka-consumer-offset-checker.sh --zookeeper localhost:2181
> --group
> >> > test-consumer-group
> >> >  or
> >> > ./kafka-consumer-offset-checker.sh --zookeeper
> >> > broker1:2181,broker2:2181,broker3:2181 --group test-consumer-group
> >> >
> >> >  DEBUG Querying X.X.X.X:9092 to locate offset manager for
> >> > test-consumer-group. (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:23:55,090] DEBUG Consumer metadata response:
> >> > ConsumerMetadataResponse(None,15,0) (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:23:55,091] DEBUG Query to X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group failed - will retry in 3000
> >> milliseconds.
> >> > (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:23:58,093] DEBUG Querying X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group. (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:23:58,102] DEBUG Consumer metadata response:
> >> > ConsumerMetadataResponse(None,15,0) (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:23:58,103] DEBUG Query to X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group failed - will retry in 3000
> >> milliseconds.
> >> > (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:01,107] DEBUG Querying X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group. (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:01,115] DEBUG Consumer metadata response:
> >> > ConsumerMetadataResponse(None,15,0) (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:01,116] DEBUG Query to X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group failed - will retry in 3000
> >> milliseconds.
> >> > (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:04,119] DEBUG Querying X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group. (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:04,124] DEBUG Consumer metadata response:
> >> > ConsumerMetadataResponse(None,15,0) (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:04,126] DEBUG Query to X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group failed - will retry in 3000
> >> milliseconds.
> >> > (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:04,993] DEBUG Got ping response for sessionid:
> >> > 0x14d33e7fbc80002 after 3ms (org.apache.zookeeper.ClientCnxn)
> >> > [2015-05-08 10:24:07,127] DEBUG Querying X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group. (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:07,131] DEBUG Consumer metadata response:
> >> > ConsumerMetadataResponse(None,15,0) (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:07,132] DEBUG Query to X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group failed - will retry in 3000
> >> milliseconds.
> >> > (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:10,132] DEBUG Querying X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group. (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:10,138] DEBUG Consumer metadata response:
> >> > ConsumerMetadataResponse(None,15,0) (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:10,139] DEBUG Query to X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group failed - will retry in 3000
> >> milliseconds.
> >> > (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:13,143] DEBUG Querying X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group. (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:13,151] DEBUG Consumer metadata response:
> >> > ConsumerMetadataResponse(None,15,0) (kafka.client.ClientUtils$)
> >> > [2015-05-08 10:24:13,152] DEBUG Query to X.X.X.X:9092 to locate offset
> >> > manager for test-consumer-group failed - will retry in 3000
> >> milliseconds.
> >> > (kafka.client.ClientUtils$)
> >> >
> >> > Thanks,
> >> > Meghana
> >> >
> >>
> >>
> >>
> >> --
> >> -Regards,
> >> Mayuresh R. Gharat
> >> (862) 250-7125
> >>
> >
> >
> >
> > --
> > *Meghana Narasimhan*
> > *Software Development DevOps Engineer*
> > *o* 919.827.4318
> > *e *mnarasimhan@bandwidth.com
> >
>
>
>
> --
> *Meghana Narasimhan*
> *Software Development DevOps Engineer*
> *o* 919.827.4318
> *e *mnarasimhan@bandwidth.com
>



-- 
-Regards,
Mayuresh R. Gharat
(862) 250-7125

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message