kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Yang <yuyan...@gmail.com>
Subject slow log segment loading during kafka 0.10.2 -> 1.0.1 upgrade
Date Sat, 10 Mar 2018 18:33:51 GMT
Hi,

We are experimenting upgrading our kafka service from 0.10.2 to 1.0.1, and
noticed that it was slow in restarting the broker after updating the kafka
binary.  On a test cluster that have ~120G data on each broker, it took
about 20 minutes to load log segments. I am wondering if there is any
work-around to reduce the long log segment loading time. Can we manually
generate snapshot files to speed up the segment loading?   Based on the
sever logs, it is also not clear to me between `loadSegments()`  and `
loadProducerState(...)`  ,which one took more time during the log segment
loading.

Thanks!

Regards,
-Yu

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message