kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ewen Cheslack-Postava <e...@confluent.io>
Subject Re: Does offsetsForTimes use createtime of logsegment file?
Date Fri, 06 Jan 2017 21:26:49 GMT
It would return the earlier one, offset 0.

-Ewen

On Thu, Jan 5, 2017 at 10:15 PM, Vignesh <vignesh.viki@gmail.com> wrote:

> Thanks. I didn't realize ListOffsetRequestV1 is only available 0.10.1
> (which has KIP-33, time index).
> When timestamp is set by user (CreationTime), and it is not always
> increasing, would this method still return the offset of first message with
> timestamp greater than equal to the provided timestamp?
>
>
> For example, in below scenario
>
> Message1, Timestamp = T1, Offset = 0
> Message2, Timestamp = T0 (or T2), Offset = 1
> Message3, Timestamp = T1, Offset = 2
>
>
> Would offsetForTimestamp(T1) return offset for earliest message with
> timestamp T1 (i.e. Offset 0 in above example) ?
>
>
> -Vignesh.
>
> On Thu, Jan 5, 2017 at 8:19 PM, Ewen Cheslack-Postava <ewen@confluent.io>
> wrote:
>
> > On Wed, Jan 4, 2017 at 11:54 PM, Vignesh <vignesh.viki@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > offsetsForTimes
> > > <https://kafka.apache.org/0101/javadoc/org/apache/kafka/
> > clients/consumer/
> > > KafkaConsumer.html#offsetsForTimes(java.util.Map)>
> > > function
> > > returns offset for a given timestamp. Does it use message's timestamp
> > > (which could be LogAppendTime or set by user) or creation time of
> > > logsegment file?
> > >
> > >
> > This is actually tied to how the ListOffsetsRequest is handled. But if
> > you're on a recent version, then the KIP
> > https://cwiki.apache.org/confluence/pages/viewpage.
> action?pageId=65868090
> > made it use the more accurate version based on message timestamps.
> >
> >
> > >
> > > KIP-33
> > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 33+-+Add+a+time+based+log+index>
> > > adds timestamp based index, and it is available only from 0.10.1 . Does
> > >  above function work on 0.10.0 ? If so, are there any differences in
> how
> > it
> > > works between versions 0.10.0 and 0.10.1 ?
> > >
> > >
> > The KIP was only adopted and implemented in 0.10.1+. It is not available
> in
> > 0.10.0.
> >
> >
> > > Thanks,
> > > Vignesh.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message