kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dragos Manolescu <Dragos.Manole...@servicenow.com>
Subject Re: ConsumerRebalanceFailedException
Date Wed, 13 Mar 2013 01:01:32 GMT
Hu Jun --

I appreciate the fast turnaround. I use Zookeeper 3.4.5.

I am not sure about the consumer GC: the exception happens when I fire up
the client's VM which shouldn't be a busy time for the GC--I've been using
autooffset.reset=largest so I assume that the client doesn't do a whole
lot upon starting up. Perhaps I am wrong about that?

Thanks,

-Dragos


On 3/12/13 11:53 AM, "Jun Rao" <junrao@gmail.com> wrote:

>What's the ZK version? You should use 3.3.4 or above, which fixed some
>bugs
>that could cause rebalance to fail.
>
> Also, a typical reason for many rebalancing is the consumer side GC. If
>so, you will see logs saying sth like "expired session" for ZK.
>
>Thanks,
>
>Jun
>
>
>On Tue, Mar 12, 2013 at 9:47 AM, Dragos Manolescu <
>Dragos.Manolescu@servicenow.com> wrote:
>
>> Hi --
>>
>> I have a Scala Kafka consumer, written in the image of the Java code
>>from
>> the Quickstart doc. I use the consumer for prototyping and debugging so
>>I
>> start and stop it quite frequently. I noticed that every now and then,
>>upon
>> starting the consumer, a ConsumerRebalanceFailedException is thrown (see
>> below). What's causing the problem this exception is signaling and what
>>are
>> the recommended practices for avoiding it? I am using Kafka 0.7.2 that I
>> built with Scala 2.9.2.
>>
>> Thanks,
>>
>> -Dragos
>>
>> Exception in thread "main"
>>kafka.common.ConsumerRebalanceFailedException:
>> Kconsumer-group_mseac02jh0rjdkq4-1363106041719-5b75c065 can't rebalance
>> after 4 retries
>> at
>> 
>>kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedReba
>>lance(ZookeeperConsumerConnector.scala:473)
>> at
>> 
>>kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsume
>>rConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:733)
>> at
>> 
>>kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnec
>>tor.scala:207)
>> at
>> 
>>kafka.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperC
>>onsumerConnector.scala:137)
>> at
>> 
>>com.servicenow.syseng.kafka.KConsumer$delayedInit$body.apply(KConsumer.sc
>>ala:69)
>> at scala.Function0$class.apply$mcV$sp(Function0.scala:34)
>> at 
>>scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
>> at scala.App$$anonfun$main$1.apply(App.scala:60)
>> at scala.App$$anonfun$main$1.apply(App.scala:60)
>> at
>> 
>>scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scal
>>a:59)
>> at scala.collection.immutable.List.foreach(List.scala:76)
>> at
>> 
>>scala.collection.generic.TraversableForwarder$class.foreach(TraversableFo
>>rwarder.scala:30)
>> at scala.App$class.main(App.scala:60)
>>
>>
>>


Mime
View raw message