kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: What is the impact of setting the consumer config max.poll.interval.ms to high value
Date Thu, 04 May 2017 23:02:08 GMT
If your consumer fails (ie. whole process dies) setting the value high
is not a problem (because the heartbeat thread dies, too, and the
failure will be detected quickly).

It's only a problem if you "main processing thread" dies (and everything
else is still up and running), or if you main processing thread has a
bug and ends up in an infinite loop not making any progress anymore. For
this case, the problem would only be detected late (ie, after 10/15
minutes).

Besides this late detention issue, there is no problem setting this
value high. For example, Streams API, sets this value to infinite (ie,
Integer.MAX_VALUE) -- even if I would not recommend this value for a
plain consumer application!

See details in the KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread



-Matthias

On 5/3/17 10:24 PM, chaitanya puchakayala wrote:
> What will be the impact of setting the consumer config max.poll.interval.ms
> to 10 or 15 mins?
> 
> How will impact nodes joining , nodes leaving, nodes crashing.?
> 
> Chaitanya
> 


Mime
View raw message