kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "M. Manna" <manme...@gmail.com>
Subject Re: Kafka upgrade process details
Date Thu, 09 May 2019 10:19:29 GMT

The upgrade process is such that

1) You ensure that there is a hard-check on protocol version if not exists
already. As you have already mentioned above, in #3 - it's to ensure that
min verson for msg formats are being adhered to before upgrade.
2) broker protocol version is to ensure that when you do rolling upgrade
there is a minimum compatibility between inter-broker comms (if that makes

API versions are there for a specific reason (to ensure compatbility is
maintained first before the protocol/msg formats are fixed). Also, it
ensures that a regular upgrade (e.g. from bugfix->major release) happens
seamlessly via rolling restart. If you want to get techincal about it, you
can look at kafka Table load and config load in github codebase.

Once you've initiated a rolling restart by setting
inter.broker.protocl.version and log.messge.format.version there is no way
back - but i am happy to be proven wrong as I have only done rolling
restart successfully (never needed to go back :) )

On Thu, 9 May 2019 at 09:54, Chylek, Artur <Artur.Chylek@sabre.com> wrote:

> Hi,
> I read the documentation about upgrading Kafka(
> http://kafka.apache.org/21/documentation.html#upgrade_2_1_0) but I have
> questions that I believe the documentation doesn't cover. I am planning to
> upgrade Kafka from 2.0.0 to 2.1.0 and would like to make sure what to do
> when something goes wrong - I am mostly interested in reverting back to the
> older version. I admit that I am confused about API versions, log message
> versions and inter broker protocol versions.
> According to the documentation these are the steps I need to follow:
>   1.  Set inter.broker.protocol.version to 2.0.0
>   2.  I don't have currently log.message.format.version set to 2.0.0, so
> according to documentation I don't need to specify it
>   3.  Upgrade broker code and restart it. This is I am not sure what
> happens next. Since I don't have log.message.format.version set then
> broker's default value will be used - I guess 2.1.1. Does it mean that:
>      *   New messages that arrive from producers will be saved in a log
> file with a new format?
>      *   Old messages that already exist in log files will be converted to
> the new format once the broker starts?
>      *   Does inter.broker.protocol.version has anything to do with
> log.message.format.version?
> I guess a) and b) are not true since according to the documentation up to
> this point I can revert back to the older version of Kafka. Could you
> confirm?
>   1.  Change inter.broker.protocol.version to 2.1.1 and restart brokers -
> after this is done reverting back is not possible. Could you explain what
> is happening in Kafka so that reverting cannot be done?
> Regards,
> Artur

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message