kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Pilat <jrpi...@gmail.com>
Subject Unexpected behavior of log.message.format.version
Date Mon, 15 Jun 2020 20:52:11 GMT

I am on a Kafka 1.1.1 cluster,
with inter.broker.protocol.version=1.1,
but log.message.format.version=0.9.0 (magic v0)

I am planning out an upgrade of log.message.format.version, and in doing so, running some
tests locally, and have a couple questions:

The behavior I see is, when an updated (2.0+) producer sends a message to a topic-partition
led by a broker with log.message.format.version=1.1, the message is written to disk with magic
v2.  This is to be expected, because that is what the setting does.

What is unexpected, however, is that the other replicas, present on brokers with log.message.format.version=0.9.0,
also write their replicas to their disks with magic v2.  The behavior I expected, was that
these follower replicas write to disk in the format specified by log.message.format.version.

Question 1: Is this behavior intended?
Question 2: If so, what is the rollback process, given that one cannot simply delete the log
segments on the 'updated' broker, change the setting back, and let the deleted segments repopulate
from the never-updated brokers?

Thanks for your time and attention,
- Jordan

View raw message