kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zhao weinan <xcvisc...@gmail.com>
Subject high level consumer api blocked forever
Date Tue, 05 Aug 2014 03:48:42 GMT
Hi, every one.

I got into a strange case that my consumer using high level api worked fine
at first, but couple days later blocked in ConsumerIterator.hasNext(),
while there are pending messages on the topic: with
kafka-console-consumer.sh I can see continuous messages.

Then i connect to consumer process's jdwp port with eclipse, suspend the
consumer thread and ConsumerFetcherThread, found ConsumerIterator blocked
on channel.take, and ConsumerFetcherThread blocked on
PartitionTopicInfo.chunkQueue.put, but channel and chunkQueue are different
object... So ConsumerFetcherThread trying to put a full LinkedBlockingQueue
while ConsumerIterator trying to take a empty LinkedBlockingQueue.  Even
more stranger thing is, with one topic has three partitions, the 3
PartitionTopicInfo.chunkQueue and ConsumerIterator.channel are 4 different
objects.

And this case is pretty frequency, has anyone encountered this, or it's
this a known issue? Any information will be helpful.

Thanks a lot!




-- 
Zhao Weinan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message