kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Balthasar Schopman <b.schop...@tech.leaseweb.com>
Subject Rebalance events when auto commit is disabled in 0.9 consumer
Date Tue, 26 Apr 2016 09:11:50 GMT
Hi,

Per consumer group we have many consumers that aggregate data for a predefined amount of time
and then write the aggregated data to a db. We're looking to set enable.auto.commit to false
and write the offsets only when the aggregated data has been written to db. In that context
we're not sure what happens when a rebalance occurs.

The rebalance can occur at any point in time, e.g. when a new consumer joins the party. What
will happen with the existing consumer at such a moment? It is likely that they have consumed
messages from the partitions that they were owning and suddenly they will own different partitions.

What does the Kafka 0.9 client do if we don't provide a ConsumerRebalanceListener?

Will we have to implement a ConsumerRebalanceListener to act on such an event?

Kind regards,

Balthasar Schopman
DevOps Software Engineer
LeaseWeb Technologies B.V.

T: +31 20 316 0232
M:
E: b.schopman@tech.leaseweb.com
W: http://www.leaseweb.com

Luttenbergweg 8, 1101 EC Amsterdam, Netherlands


Mime
View raw message