kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mathias Herberts <mathias.herbe...@gmail.com>
Subject Offset retrieval
Date Mon, 03 Oct 2011 19:22:38 GMT
In some use cases it would be useful to be able to retrieve the last
committed offsets of a connector and also be able to force those
offsets at a later time (for example when restarting an app).

Typical usecase is index creation. Imagine you have a Lucene index
which you are building. At (index) commit time, you add commit
metadata which includes the current offsets. Should the app crash, you
can retrieve the metadata from the last index commit and could start
consuming messages right where you last stopped (i.e. at the time of
last index commit).

Is there any way to do that at the moment? Maybe fiddling with znodes directly?


View raw message