kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aki <jack-ak...@marmelandia.com>
Subject Get last snapshot from compacted topic
Date Tue, 04 Aug 2015 10:16:02 GMT
I'd like to save a snapshot of a processing node's state in a compacted kafka topic. A large
number of nodes would save their snapshots in the same partition. 

What is an efficient way for a (restarted) node to find the offset of its latest snapshot?
Using just Kafka (no database, local file, etc.), is there a more efficient way than to consume
the partition from the earliest available offset (potentially reading a lot of snapshots of
other nodes).

View raw message