kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gürkan Oluç <gurkano...@gmail.com>
Subject Consumer starts to consume after changing group id
Date Wed, 15 Jan 2014 18:58:19 GMT
Hello,

We are using Kafka 0.8 with Storm. To consume from Kafka, we are using
https://github.com/wurstmeister/storm-kafka-0.8-plus this spout. We are
creating spout with this configuration :
https://gist.github.com/gurkanoluc/0cab49b8ddcfe15df8e9

Our problem is, consumers stops after working some time. From storm logs, I
am seeing that consumer offset is increasing. But, Storm is not emitting
any thing to other bolts.

Also, we had almost same issue with another classical java application. It
stops after some time, but when I change the group id of consumer group or
restart process it's starting to consume again. Changing group id sometimes
work for storm spout. My consumer configuration for this classic app is
https://gist.github.com/gurkanoluc/da46e7df263cfaffca16

I have looked to
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3Fthis
topic from topic but, It doesn't help me. Is there anything to debug
this issue?

Instead of consuming with storm, If I consume with kafka's own scripts to
consume. It consumes successfully.

Thanks for your help.
Best,

-- 
Gürkan OLUÇ
http://www.gurkanoluc.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message