kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raimar Falke" <raimar.fa...@optivo.de>
Subject Offset not written back to zookeeper
Date Mon, 29 Oct 2012 14:34:45 GMT

We are using a modified version of kafka 0.7.1 (https://github.com/optivo-org/kafka)
and had at least one case where the consumers of the group got events but the offsets
were not written back to zookeeper. The consumers stopped updating the offsets after
a release where the consumers of the group are each individually is restarted. Here 
a lot of rebalancing was taking place. After a further restart of all consumers the 
events were processed a second time. No related warning or error message could be 

Is this a known problem? What could be the cause for this?

Tracing the code paths it seems weird, that ZookeeperConsumerConnector.commitOffsets() 
does not issue a warning if no offsets were written. Is it possible that a rebalancing
was not finished and as such events were processed but without an ownership no updates
take place?

optivo GmbH
Wallstraße 16
10179 Berlin 
Telefon: 030 / 76 80 78 163
Fax:     030 / 76 80 78 499
E-Mail:  raimar.falke@optivo.de
Website: http://www.optivo.de
Handelsregister: HRB Berlin 88738
Geschäftsführer: Peter Romianowski, Ulf Richter

View raw message