kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xin Li <Xin...@trivago.com>
Subject Re: How to always consume from latest offset in kafka-streams
Date Mon, 22 Jan 2018 19:01:10 GMT
This?
consumer.auto.offset.reset = latest

Best,
Xin

On 19.01.18, 19:34, "Saloni Vithalani" <saloniv@thoughtworks.com> wrote:

    Our requirement is such that if a kafka-stream app is consuming a
    partition, it should start it's consumption from latest offset of that
    partition.
    
    This seems like do-able using
    
    streamsConfiguration.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest")
    
    Now, let's say using above configuration, the kafka-stream app started
    consuming data from latest offset for a partition. And after some time, the
    app crashes. When the app comes back live, we want it to consume data from
    the latest offset of that partition, instead of the where it left last
    reading.
    
    But I can't find anything that can help achieve it using kafka-streams api.
    
    P.S. We are using kafka-1.0.0.
    Saloni Vithalani
    Developer
    Email saloniv@thoughtworks.com
    Telephone +91 8552889571 <8552889571>
    [image: ThoughtWorks]
    <http://www.thoughtworks.com/?utm_campaign=saloni-vithalani-signature&utm_medium=email&utm_source=thoughtworks-email-signature-generator>
    

Mime
View raw message