kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit K <amitk....@gmail.com>
Subject Re: Re-consume messages
Date Mon, 07 Nov 2016 12:58:06 GMT
Thanks for your reply.

Will give it a shot.

On Sun, Nov 6, 2016 at 9:38 PM, Becket Qin <becket.qin@gmail.com> wrote:

> Hi Amit,
>
> In Kafka 0.9, the closest approach would be use
> SimpleConsumer.getOffsetBefore() to search the offset by timestamp.  And
> then you can consume from the returned biggest offset. Notice that the
> search is at log segment level and the result may not be accurate if the
> partition has been moved. In the worst case, you may consume a lot more
> messages than you want to, but you should not miss any messages.
>
> Jiangjie (Becket) Qin
>
> On Sun, Nov 6, 2016 at 1:30 AM, Amit K <amitk.abk@gmail.com> wrote:
>
> > Hi All,
> >
> > I am using kafka 0.9.0.1 with high level java producer and consumer.
> > I need to handle a case wherein I need to re-consume the already consumed
> > (and processed) messages say for last 5 days (configurable).
> >
> > Is there any way of achieving the same apart from identifying the offsets
> > for the partition (for given topic) and process the same?
> >
> > For the records, I am not storing the offsets processed.
> >
> > Thanks
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message