kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Corley <davidcor...@gmail.com>
Subject No removal of old replicas following re-balance in 0.8.0?
Date Thu, 04 Aug 2016 16:53:22 GMT
Today following a re-balance operation in our 0.8.0 cluster , we had a
bunch of disks fill up, even though our retention property values are
designed to prevent that level of disk usage.

When we investigated, it appeared the following had occurred:
1. Broker A was in the replica list for partitions 1,2,3
2. We issued a re-balance per the re-assign partition tool -
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-6.ReassignPartitionsTool
3. Wait for the the new replica / leader to join the ISR
4. In this case, Broker A was neither a leader nor replica for any of
partitions 1,2,3 after the re-balance, yet the data files for those
partitions remained on Broker A's data folder after the re-balance had
completed.

Is this expected behavior? Are the segments remaining on Broker A, but no
longer used by the cluster going to lie around forever? Are they subject
only to the time retention policy from that point on?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message