kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saloni Vithalani <salo...@thoughtworks.com>
Subject How to always consume from latest offset in kafka-streams
Date Fri, 19 Jan 2018 17:23:47 GMT
Our requirement is such that if a kafka-stream app is consuming a
partition, it should start it's consumption from latest offset of that
partition.

This seems like do-able using

streamsConfiguration.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest")

Now, let's say using above configuration, the kafka-stream app started
consuming data from latest offset for a partition. And after some time, the
app crashes. When the app comes back live, we want it to consume data from
the latest offset of that partition, instead of the where it left last
reading.

But I can't find anything that can help achieve it using kafka-streams api.

P.S. We are using kafka-1.0.0.
Saloni Vithalani
Developer
Email saloniv@thoughtworks.com
Telephone +91 8552889571 <8552889571>
[image: ThoughtWorks]
<http://www.thoughtworks.com/?utm_campaign=saloni-vithalani-signature&utm_medium=email&utm_source=thoughtworks-email-signature-generator>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message