kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Dubiel <dubiel.a...@gmail.com>
Subject Re: querying messages based on timestamps
Date Tue, 30 Jun 2015 20:16:33 GMT
We faced similar problem and ended up with implementing variant of golden
section search, that reads message using simple consumer and checks the
timestamp (timestamps are appended by our producer though, they do not come
from any Kafka metadata) till it finds message closest to given date.

Adam

2015-06-30 21:52 GMT+02:00 Shushant Arora <shushantarora09@gmail.com>:

> Is it possible using low level consumer to get kafka messages based on
> timestamp, say I want to get all messages from last 5 minutes. I don't know
> what were offsets of partitions 5 minutes back.
>
> In low level consumer : when I gave epoch for  whichTime , it failed.
>  requestInfo.put(topicAndPartition, new
> PartitionOffsetRequestInfo(whichTime,
> 1));
>
> Is only latest and earliest supported in timestamp,Is there any way to
> filter messages based on timestamp?
>
> Thanks
> Shushant
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message