kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mathias Söderberg <mathias.soederb...@gmail.com>
Subject Questions re: partition reassignment
Date Sun, 21 Sep 2014 21:32:56 GMT
Good day,

We're currently running a Kafka cluster in our staging environment, testing
everything out and so far it runs fairly smoothly.

One thing that we're currently looking into is scaling a cluster that is
already up 'n serving data, and I have some questions regarding moving
partitions between brokers.

My current interpretation is that if I start a partition reassignment, for
the sake of simplicity let's assume it's just for a single partition, the
new leader will first become a follower of the current leader, and when it
has caught up it'll transfer leadership over to itself?
If so, is Kafka only concerned about data that has been flushed to disk? So
if there's data that hasn't been flushed to disk yet it'll be discarded
when the new leader gains leadership?

What happens if neither of the new ARs are part of the previous AR set? I
suppose that they'll all be followers and when the new leader has caught up
it'll assume leadership?

I've played around with controlled shutdown and noticed that the broker
that's shutting down is truncating logs prior to actually shutting down,
which I don't really understand? I cannot really tell whether it does this
for all topics or just a subset of the ones that it's AR for as we have
quite a lot of topics, but I figured I'd just throw it out there, as I feel
like it could be explained by the answers to the questions above.

Should mention that we're running v0.8.1.1 with mostly stock configuration,
and that our producers are currently using ACK = 1 as well.

Best regards,

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message