kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brett Rann <br...@zendesk.com.INVALID>
Subject Re: Problems with broker upgrade from 1.1.0 to 2.0.0
Date Tue, 09 Oct 2018 01:26:16 GMT
I'd

* verify you shutdown cleanly when restarting the broker
* test restarting the broker first before upgrading it (it may have been
sitting on a corrupt file for a long time and the issue is unrelated to the
upgrade)

re your questions:

1) probably not, unless it was caused by an unclean shutdown, in which case
shutdown cleanly. :)
2) probably not, since the indexes are rebuilt.

FWIW i've done a bunch of 1.1.0 -> 2.0 upgrades and haven't had this issue.
For sure have seen it in the past though.


On Tue, Oct 9, 2018 at 2:09 AM Claudia Wegmann <c.wegmann@kasasi.de> wrote:

> Dear community,
>
> I updated my kafka cluster from version 1.1.0 to 2.0.0 according to the
> upgrade guide for rolling upgrades today. I encountered a problem after
> starting the new broker version. The log is full of
> "Found a corrupted index file corresponding to log file
> /data/kafka-logs/resultGatewayDataDispatcher-22/00000000000002858226.log
> due to Corrupt time index found, time index file
> (/data/kafka-logs/resultGatewayDataDispatcher-22/00000000000002858226.timeindex)
> has non-zero size but the last timestamp is 0 which is less than the first
> timestamp 1538583935827}, recovering segment and rebuilding index files...
> (kafka.log.Log)"
>
> Said recovering takes quite a bit of time.
>
> Now I'm not too keen to do the broker update in the production environment
> because of the relative long processing pause.
> The questions are:
> 1.) Is there a way to avoid this error and therefore make starting the new
> broker faster?
> 2.) Does the error imply any data loss?
>
> Any help is apricated 😊
> Best,
> Claudia
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message