kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jack Lund <jack.l...@braintreepayments.com>
Subject Under-replicated Partitions while rolling Kafka nodes in AWS
Date Thu, 05 Jan 2017 15:55:53 GMT
Hello, all.

We're running multiple Kafka clusters in AWS, and thus multiple Zookeeper
clusters as well. When we roll out changes to our zookeeper nodes (which
involves changes to the AMI, which means terminating the zookeeper instance
and bringing up a new one in its place) we have to restart our Kafka
brokers one at a time so they can pick up the new zookeeper IP address.

What we've noticed is that, as the brokers are restarted, we get alerts for
under-replicated partitions, which seems strange since it seems like the
shutdown process should take care of moving any replicas and the leadership
election process.

This is causing us some pain because it means that we get pages whenever we
roll out changes to Zookeeper.

Does anybody have any ideas why this would be happening, and how we can
avoid it?

Thanks.

-Jack Lund
 Braintree Payments

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message