kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prachi Govalkar <pgoval...@box.com.INVALID>
Subject Question regarding upgrade
Date Thu, 18 Oct 2018 00:22:36 GMT
Hi,
We have a cross dc replication setup for our kafka cluster. dc1 replicates
to dc2 and dc2 replicates to dc1.
We did a inplace rolling upgrade for our kafka cluster (dc1) version
0.8.1.1 to 1.1.1 by updating the below parameters in server.properties file
before the upgrade.

   - inter.broker.protocol.version=0.8.1.1
   - log.message.format.version=0.8.1.1


Kafka was working fine after the upgrade, then we changed the below params
to 1.1.1 and then the cross dc replication stopped

   - inter.broker.protocol.version=1.1.1
   - log.message.format.version=1.1.1

We reverted the log.message.format.version=0.8.1.1 version and again the
cross dc replication started, but few of the partitions were not getting
processed in the dc1 which was upgraded to kafka version 1.1.1.
We then reverted the log.message.format.version=1.1.1, the partitions which
were not getting processed earlier started processing but the partitions
which were processing stopped.
We downgraded the kafka again  to 0.8.1.1 version, but the topics were not
getting consumed.
there are no error logs in kafka or consumer end.
However we think due to changing log.message.format.version multiple times
the data must have corrupted

Can you share some solution to prevent data loss.

Prachi Govalkar
M. 650-847-9496


[image: box] <http://www.box.com/home?from=em>
Cloud Content Management
box.com <http://www.box.com/home?from=em>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message