storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Kania <>
Subject Storm Kafka retention of in progress messages
Date Mon, 08 Aug 2016 14:36:55 GMT
I am wondering how long the Kafka Spout retains a reference to messages that are in progress
but have not been acked. I have a situation where I am getting no failed messages in the UI
but getting many retries when the topology is restarted. I am wondering if the Kafka Spout
might be forgetting about these messages by the time the ack is coming back. The processing
of the message can take up to 5 hours in some cases so the timeout is set to this duration.
View raw message