kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ye Hong <yeho...@gmail.com>
Subject Kafka partition reassignment causes replica number and Isr to go above configured value
Date Wed, 05 Aug 2015 19:02:41 GMT
We recently performed a partition reassignment.
Our Kafka cluster is on 0.8.1.1, and all topics are configured to have 2
replicas.

After the completion of reassignment, we have noticed that some of the
partitions has 3 replicas and 3 Isrs.

for example:
    Topic: topic1    Partition: 7    Leader: 5    Replicas: 5,2    Isr: 5,2
    Topic: topic1    Partition: 8    Leader: 6    Replicas: 6,4,2    Isr:
6,4,2
    Topic: topic1    Partition: 32    Leader: 5    Replicas: 5,4,2    Isr:
5,4,2

Previously, we have seen 3 replicas during the reassignment, however, the
Isr count was always 2, and replica count will drop back to 2 after the
completion of reassignment.

Just wonder if someone has encountered same problem.
In terms of fixing, I am thinking about triggering a partial partition
reassignment with a manually created partition assignment config that would
bring the replicas back to 2. Is this the right way to fix such an issue?

Thanks,

Ye

P.S.
Here are the step we did for the reassignment:
1. /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper
<zookeeper_host_name>:<port> --broker-list "<broker_list>" --generate
--topics-to-move-json-file <topics-to-move_json_file>
2. From the console output, copy the second part and put it into a text
file assignment_json_file.
3. /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper
<zookeeper_host_name>:<port> --broker-list "<broker_list>" --execute
--topics-to-move-json-file <topics-to-move_json_file>
--reassignment-json-file <assignment_json_file>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message