kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wenbing shen <oliver.shen...@gmail.com>
Subject Kafka shuts down abnormally and restarts, always taking more than a few hours to load
Date Tue, 13 Oct 2020 06:52:44 GMT
   -
      -
         -
            -

            -

             Kafka shuts down abnormally and restarts, always taking more
            than a few hours to load.
            -

            We used Kafka-2.0.0,we tried to turn it
            up num.recovery.threads.per.data.dir,but the startup
process is always
            limited by the slow loading of partitions.We observed a
partition that took
            two hours to load.
            -

            The action of the partition in the startup process is mainly
            Loading producer state from snapshot file and writing
producer snapshotat
            offset.
            -

            A lot of time is spent in rebuildProducerState and
            segment.recover methods.
            -

            The cluster consists of 21 nodes, each with about 1500
            partitions and 700 partition leaders.
            -

            Is there any way to reduce the startup time of Kafka when it
            shuts down abnormally?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message