storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nitin Gupta <>
Subject Offset not updated in Zookeeper
Date Thu, 02 Jun 2016 14:01:56 GMT
Hi Guys,

I am facing a strange issue. I am using KafkaSpout and BaseBasicBolt to process messages from
the Kafka.
There are around 1600 messages in the queue at a given point. From the worker log it seems
the messages are being picked but when I use the Zookeeper shell to check the offset being
consumed by the topology it gives the same offset(148 ) for a long period of time (checked
for 1 hour).

The average latency time being shown on the Storm UI is 10 seconds.

I would like to know after what duration the KafkaSpout would write the offset being consumed
back to the Zookeeper. Should I move to BaseRickBolt and do an explicit ack so that the offset
are written back to the Zookeeper.

Thanks & Regards,
Nitin Gupta

View raw message