kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zhao weinan <xcvisc...@gmail.com>
Subject Re: high level consumer api blocked forever
Date Tue, 05 Aug 2014 07:45:59 GMT
Hi, every one.

forget it, sorry it's my fault. when my thread throwed a exception i caught
it and then call "createJavaConsumerConnector" with out close old one...
then of course cause redundant consumers in consumer group some of which
will never get messages.


2014-08-05 11:48 GMT+08:00 zhao weinan <xcviscool@gmail.com>:

> Hi, every one.
> I got into a strange case that my consumer using high level api worked
> fine at first, but couple days later blocked in ConsumerIterator.hasNext(),
> while there are pending messages on the topic: with
> kafka-console-consumer.sh I can see continuous messages.
> Then i connect to consumer process's jdwp port with eclipse, suspend the
> consumer thread and ConsumerFetcherThread, found ConsumerIterator blocked
> on channel.take, and ConsumerFetcherThread blocked on
> PartitionTopicInfo.chunkQueue.put, but channel and chunkQueue are different
> object... So ConsumerFetcherThread trying to put a full LinkedBlockingQueue
> while ConsumerIterator trying to take a empty LinkedBlockingQueue.  Even
> more stranger thing is, with one topic has three partitions, the 3
> PartitionTopicInfo.chunkQueue and ConsumerIterator.channel are 4 different
> objects.
> And this case is pretty frequency, has anyone encountered this, or it's
> this a known issue? Any information will be helpful.
> Thanks a lot!
> --
> Zhao Weinan

Zhao Weinan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message