kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vignesh <vignesh.v...@gmail.com>
Subject Re: Does offsetsForTimes use createtime of logsegment file?
Date Thu, 19 Jan 2017 18:02:05 GMT
Another question, with getOffsetsBefore, we used to be able to get offsets
for time in older versions.
.10 doesn't have an equivalent method.

Is there any other way to achieve the same functionality as getOffsetsBefore
in .10 ? Does a .10 server respond to ListOffsetRequestV0 request?


On Fri, Jan 6, 2017 at 1:26 PM, Ewen Cheslack-Postava <ewen@confluent.io>
wrote:

> It would return the earlier one, offset 0.
>
> -Ewen
>
> On Thu, Jan 5, 2017 at 10:15 PM, Vignesh <vignesh.viki@gmail.com> wrote:
>
> > Thanks. I didn't realize ListOffsetRequestV1 is only available 0.10.1
> > (which has KIP-33, time index).
> > When timestamp is set by user (CreationTime), and it is not always
> > increasing, would this method still return the offset of first message
> with
> > timestamp greater than equal to the provided timestamp?
> >
> >
> > For example, in below scenario
> >
> > Message1, Timestamp = T1, Offset = 0
> > Message2, Timestamp = T0 (or T2), Offset = 1
> > Message3, Timestamp = T1, Offset = 2
> >
> >
> > Would offsetForTimestamp(T1) return offset for earliest message with
> > timestamp T1 (i.e. Offset 0 in above example) ?
> >
> >
> > -Vignesh.
> >
> > On Thu, Jan 5, 2017 at 8:19 PM, Ewen Cheslack-Postava <ewen@confluent.io
> >
> > wrote:
> >
> > > On Wed, Jan 4, 2017 at 11:54 PM, Vignesh <vignesh.viki@gmail.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > offsetsForTimes
> > > > <https://kafka.apache.org/0101/javadoc/org/apache/kafka/
> > > clients/consumer/
> > > > KafkaConsumer.html#offsetsForTimes(java.util.Map)>
> > > > function
> > > > returns offset for a given timestamp. Does it use message's timestamp
> > > > (which could be LogAppendTime or set by user) or creation time of
> > > > logsegment file?
> > > >
> > > >
> > > This is actually tied to how the ListOffsetsRequest is handled. But if
> > > you're on a recent version, then the KIP
> > > https://cwiki.apache.org/confluence/pages/viewpage.
> > action?pageId=65868090
> > > made it use the more accurate version based on message timestamps.
> > >
> > >
> > > >
> > > > KIP-33
> > > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > > 33+-+Add+a+time+based+log+index>
> > > > adds timestamp based index, and it is available only from 0.10.1 .
> Does
> > > >  above function work on 0.10.0 ? If so, are there any differences in
> > how
> > > it
> > > > works between versions 0.10.0 and 0.10.1 ?
> > > >
> > > >
> > > The KIP was only adopted and implemented in 0.10.1+. It is not
> available
> > in
> > > 0.10.0.
> > >
> > >
> > > > Thanks,
> > > > Vignesh.
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message