kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject SimpleConsumer.getOffsetsBefore() in 0.9 KafkaConsumer
Date Wed, 20 Jan 2016 11:09:09 GMT
Hi,

I'm currently looking into implementing a load shedding strategy into
Flink's Kafka consumer.

Therefore, I would like to allow users to request the latest offset of the
subscribed TopicPartitions, so that they can
a) determine the lag
b) maybe set the next fetch offset to the latest offset (or anything in
between)

With the SimpleConsumer, there is the getOffsetsBefore() method, which
allows to request the largest offset.
For the new KafkaConsumer, I only found the seekToEnd() and
seekToBeginning() methods, but they don't allow to get the actual end or
beginning.


Are there plans to add such an API to the KafkaConsumer again?
Can you suggest any workarounds?

Regards,
Robert

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message