kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sam Pegler <sam.peg...@infectiousmedia.com>
Subject Re: Lots of messages when starting Kafka
Date Wed, 06 Jul 2016 14:00:10 GMT
>If you kill Kafka with SIGTERM, then naturally it will leave behind corrupt
index files. It will safely rebuild them. Don't worry.

I think you mean SIGKILL.  SIGTERM should be handled normally an won't
result in corrupted index files at startup.

__

Sam Pegler

Production Engineer

T. +44(0) 07 562 867 486

<http://www.infectiousmedia.com/>
3-7 Herbal Hill / London / EC1R 5EJ
www.infectiousmedia.com

This email and any attachments are confidential and may also be privileged.
If you
are not the intended recipient, please notify the sender immediately, and
do not
disclose the contents to another person, use it for any purpose, or store,
or copy
the information in any medium. Please also destroy and delete the message
from
your computer.


On 6 July 2016 at 14:54, Peter Davis <davispw@gmail.com> wrote:

> If you kill Kafka with SIGTERM, then naturally it will leave behind corrupt
> index files. It will safely rebuild them. Don't worry.
>
> The Group Metadata Manager messages are normal -- that's just Kafka
> starting up.
>
> Perhaps the corrupted index log messages could be less scary, though. This
> is a very common question from new users...
>
> -Peter
>
> On Wed, Jul 6, 2016 at 06:50 Fabian Kochem <
> fabian.kochem@concepts-and-training.de> wrote:
>
> > Hi,
> >
> > this is my first time using a mailing list so please forgive me if I'm
> > doing something wrong.
> >
> > When I start Kafka, I get a ton of messages like this:
> >
> > WARN Found a corrupted index file,
> > /kafka/logs/__consumer_offsets-48/00000000000000000000.index, deleting
> > and rebuilding index... (kafka.log.Log)
> > INFO Recovering unflushed segment 0 in log __consumer_offsets-48.
> > (kafka.log.Log)
> > INFO Completed load of log __consumer_offsets-48 with log end offset 0
> > (kafka.log.Log)
> > WARN Found a corrupted index file,
> > /kafka/logs/__consumer_offsets-19/00000000000000000000.index, deleting
> > and rebuilding index... (kafka.log.Log)
> > INFO Recovering unflushed segment 0 in log __consumer_offsets-19.
> > (kafka.log.Log)
> > INFO Completed load of log __consumer_offsets-19 with log end offset 0
> > (kafka.log.Log)
> > WARN Found a corrupted index file,
> > /kafka/logs/__consumer_offsets-3/00000000000000000000.index, deleting
> > and rebuilding index... (kafka.log.Log)
> > INFO Recovering unflushed segment 0 in log __consumer_offsets-3.
> > (kafka.log.Log)
> > INFO Completed load of log __consumer_offsets-3 with log end offset 0
> > (kafka.log.Log)
> >
> > And after that, there's an equal amount of messages which look like this:
> >
> > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
> > metadata from [__consumer_offsets,22]
> > (kafka.coordinator.GroupMetadataManager)
> > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
> > from [__consumer_offsets,22] in 11 milliseconds.
> > (kafka.coordinator.GroupMetadataManager)
> > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
> > metadata from [__consumer_offsets,25]
> > (kafka.coordinator.GroupMetadataManager)
> > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
> > from [__consumer_offsets,25] in 4 milliseconds.
> > (kafka.coordinator.GroupMetadataManager)
> > INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
> > metadata from [__consumer_offsets,28]
> > (kafka.coordinator.GroupMetadataManager)
> > INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
> > from [__consumer_offsets,28] in 2 milliseconds.
> > (kafka.coordinator.GroupMetadataManager)
> >
> > Did I misconfigure something?
> >
> > I only have one topic with a few consumers, so this seems a bit much. I
> > shutdown Kafka using SIGTERM.
> >
> > Thanks in advance,
> > Fabian Kochem
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message