storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kashyap Mhaisekar <kashya...@gmail.com>
Subject Re: Kafka Rebalance exceptions in Storm
Date Thu, 03 Apr 2014 12:40:46 GMT
Thanks. How did you manage the offset persistence in this case?

On Thursday, April 3, 2014, padma priya chitturi <padmapriya30@gmail.com>
wrote:

> The same exception i encountered when tried to integrate storm spout with
> kafka high level consumer. This was resolved when i used SimpleConsumer.
>
>
> On Thu, Apr 3, 2014 at 8:26 AM, Kashyap Mhaisekar <kashyap.m@gmail.com<javascript:_e(%7B%7D,'cvml','kashyap.m@gmail.com');>
> > wrote:
>
>> Hi,
>> I consistently get the following exception when I submit a storm topology
>> using StormSubmitter. I am using 0.8.2 Storm.
>> The spout I use is a Kafka High Level consumer thats custom written.
>>
>> When I submit the same topology using LocalCluster, it works fine. Am I
>> missing anything
>>
>> *kafka.common.ConsumerRebalanceFailedException:
>> Sample_Kafka_client_abcd_com-1396492916541-2f28228c can't rebalance after 4
>> retries*
>> *        at
>> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:432)
>> ~[stormjar.jar:na]*
>> *        at
>> kafka.consumer.ZookeeperConsumerConnector.kafka$consumer$ZookeeperConsumerConnector$$reinitializeConsumer(ZookeeperConsumerConnector.scala:722)
>> ~[stormjar.jar:na]*
>> *        at
>> kafka.consumer.ZookeeperConsumerConnector.consume(ZookeeperConsumerConnector.scala:212)
>> ~[stormjar.jar:na]*
>> *        at
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:80)
>> ~[stormjar.jar:na]*
>> *        at
>> kafka.javaapi.consumer.ZookeeperConsumerConnector.createMessageStreams(ZookeeperConsumerConnector.scala:92)
>> ~[stormjar.jar:na]*
>>
>>
>> *2014-04-02 21:41:58 b.s.util [INFO] Halting process: ("Worker died") *
>>
>> Thanks in advance
>> Regards,
>> Kashyap
>>
>
>

Mime
View raw message