kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kanak Biscuitwala <kana...@hotmail.com>
Subject Partition assigner that takes current assignment into account
Date Tue, 02 Aug 2016 03:46:48 GMT

I wanted to highlight an alternative algorithm that we have written to generate JSON compatible
with kafka-reassign-partitions.sh. It's novel for the following reasons:

- Like the existing algorithm, it's rack-aware, balances partitions assigned to hosts, and
number of leader replicas on each host

- Unlike the existing algorithm, it takes the current assignment into account to minimize
data movement

- Because of the above property, it works well for decommissioning or replacing nodes, as
well as adding new nodes, and taking only a subset of partitions away from existing nodes.

Take a look: https://github.com/SiftScience/kafka-assigner

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message