kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Davis <davi...@gmail.com>
Subject Re: Lots of messages when starting Kafka
Date Wed, 06 Jul 2016 13:54:03 GMT
If you kill Kafka with SIGTERM, then naturally it will leave behind corrupt
index files. It will safely rebuild them. Don't worry.

The Group Metadata Manager messages are normal -- that's just Kafka
starting up.

Perhaps the corrupted index log messages could be less scary, though. This
is a very common question from new users...

-Peter

On Wed, Jul 6, 2016 at 06:50 Fabian Kochem <
fabian.kochem@concepts-and-training.de> wrote:

> Hi,
>
> this is my first time using a mailing list so please forgive me if I'm
> doing something wrong.
>
> When I start Kafka, I get a ton of messages like this:
>
> WARN Found a corrupted index file,
> /kafka/logs/__consumer_offsets-48/00000000000000000000.index, deleting
> and rebuilding index... (kafka.log.Log)
> INFO Recovering unflushed segment 0 in log __consumer_offsets-48.
> (kafka.log.Log)
> INFO Completed load of log __consumer_offsets-48 with log end offset 0
> (kafka.log.Log)
> WARN Found a corrupted index file,
> /kafka/logs/__consumer_offsets-19/00000000000000000000.index, deleting
> and rebuilding index... (kafka.log.Log)
> INFO Recovering unflushed segment 0 in log __consumer_offsets-19.
> (kafka.log.Log)
> INFO Completed load of log __consumer_offsets-19 with log end offset 0
> (kafka.log.Log)
> WARN Found a corrupted index file,
> /kafka/logs/__consumer_offsets-3/00000000000000000000.index, deleting
> and rebuilding index... (kafka.log.Log)
> INFO Recovering unflushed segment 0 in log __consumer_offsets-3.
> (kafka.log.Log)
> INFO Completed load of log __consumer_offsets-3 with log end offset 0
> (kafka.log.Log)
>
> And after that, there's an equal amount of messages which look like this:
>
> INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
> metadata from [__consumer_offsets,22]
> (kafka.coordinator.GroupMetadataManager)
> INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
> from [__consumer_offsets,22] in 11 milliseconds.
> (kafka.coordinator.GroupMetadataManager)
> INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
> metadata from [__consumer_offsets,25]
> (kafka.coordinator.GroupMetadataManager)
> INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
> from [__consumer_offsets,25] in 4 milliseconds.
> (kafka.coordinator.GroupMetadataManager)
> INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
> metadata from [__consumer_offsets,28]
> (kafka.coordinator.GroupMetadataManager)
> INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
> from [__consumer_offsets,28] in 2 milliseconds.
> (kafka.coordinator.GroupMetadataManager)
>
> Did I misconfigure something?
>
> I only have one topic with a few consumers, so this seems a bit much. I
> shutdown Kafka using SIGTERM.
>
> Thanks in advance,
> Fabian Kochem
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message