kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vignesh <vignesh.v...@gmail.com>
Subject Re: Does offsetsForTimes use createtime of logsegment file?
Date Fri, 06 Jan 2017 06:15:19 GMT
Thanks. I didn't realize ListOffsetRequestV1 is only available 0.10.1
(which has KIP-33, time index).
When timestamp is set by user (CreationTime), and it is not always
increasing, would this method still return the offset of first message with
timestamp greater than equal to the provided timestamp?


For example, in below scenario

Message1, Timestamp = T1, Offset = 0
Message2, Timestamp = T0 (or T2), Offset = 1
Message3, Timestamp = T1, Offset = 2


Would offsetForTimestamp(T1) return offset for earliest message with
timestamp T1 (i.e. Offset 0 in above example) ?


-Vignesh.

On Thu, Jan 5, 2017 at 8:19 PM, Ewen Cheslack-Postava <ewen@confluent.io>
wrote:

> On Wed, Jan 4, 2017 at 11:54 PM, Vignesh <vignesh.viki@gmail.com> wrote:
>
> > Hi,
> >
> > offsetsForTimes
> > <https://kafka.apache.org/0101/javadoc/org/apache/kafka/
> clients/consumer/
> > KafkaConsumer.html#offsetsForTimes(java.util.Map)>
> > function
> > returns offset for a given timestamp. Does it use message's timestamp
> > (which could be LogAppendTime or set by user) or creation time of
> > logsegment file?
> >
> >
> This is actually tied to how the ListOffsetsRequest is handled. But if
> you're on a recent version, then the KIP
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65868090
> made it use the more accurate version based on message timestamps.
>
>
> >
> > KIP-33
> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > 33+-+Add+a+time+based+log+index>
> > adds timestamp based index, and it is available only from 0.10.1 . Does
> >  above function work on 0.10.0 ? If so, are there any differences in how
> it
> > works between versions 0.10.0 and 0.10.1 ?
> >
> >
> The KIP was only adopted and implemented in 0.10.1+. It is not available in
> 0.10.0.
>
>
> > Thanks,
> > Vignesh.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message