kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Otto <o...@wikimedia.org>
Subject Re: offsetsForTimes API performance
Date Mon, 22 Jan 2018 15:24:59 GMT
Speaking of, has there been any talk of combining those two requests into a
single API call?  I’d assume that offsetForTimes + consumer seek is
probably the most common use case of offsetForTimes.  Maybe a round trip
could be avoided if the broker could just auto-assign the consumer to the
offset for a timestamp.


On Mon, Jan 22, 2018 at 9:59 AM, srimugunthan dhandapani <
srimugunthan.dhandapani@gmail.com> wrote:

>  Hi all,
>
> We use kafka as our store and  every one of our record is associated with a
> timeStamp. We pull data from kafka by seeking to a timeStamp offset
> everytime and then get the records by polling. We use KafkaConsumer's
> offsetsForTimes (
> https://kafka.apache.org/0110/javadoc/org/apache/kafka/clients/consumer/
> KafkaConsumer.html#offsetsForTimes(java.util.Map)
>
> ) API to  find offset and seek to a particular time offset.
>
>  We see that using the offsetsForTimes API and the subsequent seek to the
> offset takes anything from 17 milliseconds to 500millisec per iteration.
>
> I would like to know if anybody has done any performance testing of the
> offsetsForTimes API and what does the performance of the API depend on?
> Will the API be slower if there is more data in the kafka?
>
>
> thanks,
> mugunthan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message