kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Becket Qin <becket....@gmail.com>
Subject Re: Re-consume messages
Date Sun, 06 Nov 2016 16:08:50 GMT
Hi Amit,

In Kafka 0.9, the closest approach would be use
SimpleConsumer.getOffsetBefore() to search the offset by timestamp.  And
then you can consume from the returned biggest offset. Notice that the
search is at log segment level and the result may not be accurate if the
partition has been moved. In the worst case, you may consume a lot more
messages than you want to, but you should not miss any messages.

Jiangjie (Becket) Qin

On Sun, Nov 6, 2016 at 1:30 AM, Amit K <amitk.abk@gmail.com> wrote:

> Hi All,
>
> I am using kafka 0.9.0.1 with high level java producer and consumer.
> I need to handle a case wherein I need to re-consume the already consumed
> (and processed) messages say for last 5 days (configurable).
>
> Is there any way of achieving the same apart from identifying the offsets
> for the partition (for given topic) and process the same?
>
> For the records, I am not storing the offsets processed.
>
> Thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message