storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Birendra Kumar Singh <>
Subject Message processing from a kafka topic
Date Fri, 01 Jan 2016 18:37:42 GMT
I have a storm setup that picks up messages from a kafka topic and
processes and persists them.
I want to understand how storm gurantees message processing in such a

Consider the below scenario:
I have configured multiple supervisors+workers for a storm cluster.
The KafkaSpout is reading message from the topic and then passes on this a
bolt. The bolt acks upon completion and the spout moves forward to the next
I have 2 supervisors running - each of which are running 3 workers each.
>From what I understand - each of the worker on every supervisor is capable
to processing a message.
So, at any given time 6 messages are being processed parallely in storm
what if the second message fails, either due to worker shutdown or due to
supervisor shutdown.
the zookeeper is already pointing to the 7 message for the consumer group.
In such a scenario, how will the second message get processed?

Thanks for reading

View raw message