kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: Default commit interval for Kafka Streams
Date Tue, 16 Apr 2019 17:27:54 GMT
The commit interval is small to keep end-to-end processing latency
small. For example, if data is repartitioned, a downstream task can only
read the data after the upstream tasks commit its transaction.

-Matthias

On 4/16/19 9:56 AM, Scott Reynolds wrote:
> Hi,
> 
> I have been unable to determine  why the default commit interval for
> Exactly Once Streams application is 100L. This seems really aggressive and
> does produce a large amount of offsets to our broker.  I have changed this
> in our application but I am worried I have now introduced a bug in the
> application.
> 
> Can some provide color and context into why we use 100L ms as the commit
> interval for Exactly Once Streams api?
> 
> 1.
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-129%3A+Streams+Exactly-Once+Semantics
> 2. https://github.com/apache/kafka/pull/2945
> 


Mime
View raw message