kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Thygesen <pt.activ...@gmail.com>
Subject Shutdown/Ctrl-C and ConsumerRebalanceFailedException
Date Tue, 20 Mar 2012 09:42:11 GMT
When I shutdown my consumer with crtl-c and tries to restart it quickly
afterwards, I usually get ConsumerRebalanceFailedException (see below). The
application then seems to hang.. or at least I'm sure if it is running any
more.. If this exception is thrown, will the consumer the intelligently
wait for the rebalancing to complete? and then resume consumption?

I found a page https://cwiki.apache.org/KAFKA/consumer-co-ordinator.htmlthat
describes something about Consumer Co-ordinator.. according to this
the consumer
group remains in this state until the next rebalancing attempt is
triggered. But when is it triggered?

Could a shutdown hock with a consumer.commitOffsets help?
Does the consumer.shutdown implicit commitOffsets?


Exception in thread "main" kafka.common.ConsumerRebalanceFailedException:
contentItem-consumer-group-1_cphhdfs01node09-1332175323213-e6a3010f can't
rebalance after 4 retries
        at
kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:467)
        at
kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:204)
        at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:75)
        at
kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:89)
        at
com.infopaq.research.repository.uima.ContentItemClient.consume(ContentItemClient.java:75)
        at
com.infopaq.research.repository.uima.ContentItemClient.main(ContentItemClient.java:111)


Brgds,
Peter Thygesen

BTW: Great work, very interesting project.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message