storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milind Vaidya <kava...@gmail.com>
Subject Getting Kafka Offset in Storm Bolt
Date Wed, 04 May 2016 18:31:47 GMT
Is there any way I can know what Kafka offset corresponds to current tuple
I am processing in a bolt ?

Use case : Need to batch events from Kafka, persists them to a local file
and eventually upload it to the S3. To manager failure cases, need to know
the Kafka offset for a message, so that it can be persisted to Zookeeper
and will be used to write / upload file.

Mime
View raw message