storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hugo Da Cruz Louro <hlo...@hortonworks.com>
Subject Re: Kafka Spout enable.auto.commit=false
Date Tue, 21 Feb 2017 18:01:35 GMT
As per KafkaConsumer documentation<https://kafka.apache.org/0100/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html>:

“ Setting enable.auto.commit=true means that offsets are committed automatically with a
frequency controlled by the config auto.commit.interval.ms. “

That means that the Spout disregards if the Tuple is acked or not;  every time interval (as
controlled by auto.commit.interval.ms ) the offset will be committed. This can have have an
impact on the delivery guarantees, because an offset may be committed, yet the tuple may fail.

On Feb 20, 2017, at 8:15 AM, Igor Kuzmenko <f1sherox@gmail.com<mailto:f1sherox@gmail.com>>
wrote:

Hello, I'd like to understand difference between auto commit mode true/false in new KafkaSpout.

With auto.commit.enabled = false KafkaSpout will move my offset relying on acked tuples, it
seems easy.

But what happens if I turn auto commit on?
How Kafka make decision which offset to commit?

Mime
View raw message