kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Kochem <fabian.koc...@concepts-and-training.de>
Subject Re: Lots of messages when starting Kafka
Date Wed, 06 Jul 2016 14:09:31 GMT
I use "docker stop" which sends SIGTERM and then waits 10 seconds before 
it sends SIGKILL.
I assume it doesn't get to send SIGKILL because when I don't use Docker, 
a usual Ctrl+C stops Kafka way faster than 10 seconds.


On 06/07/16 16:05, Peter Davis wrote:
> You're right Sam. Getting my signals confused before coffee.
>
> Fabian, how did you in fact kill Kafka? "kill"? Ctrl+C?
>
> -Peter
>
> On Wed, Jul 6, 2016 at 07:00 Sam Pegler <sam.pegler@infectiousmedia.com>
> wrote:
>
>>> If you kill Kafka with SIGTERM, then naturally it will leave behind
>> corrupt
>> index files. It will safely rebuild them. Don't worry.
>>
>> I think you mean SIGKILL.  SIGTERM should be handled normally an won't
>> result in corrupted index files at startup.
>>
>> __
>>
>> Sam Pegler
>>
>> Production Engineer
>>
>> T. +44(0) 07 562 867 486
>>
>> <http://www.infectiousmedia.com/>
>> 3-7 Herbal Hill / London / EC1R 5EJ
>> www.infectiousmedia.com
>>
>> This email and any attachments are confidential and may also be privileged.
>> If you
>> are not the intended recipient, please notify the sender immediately, and
>> do not
>> disclose the contents to another person, use it for any purpose, or store,
>> or copy
>> the information in any medium. Please also destroy and delete the message
>> from
>> your computer.
>>
>>
>> On 6 July 2016 at 14:54, Peter Davis <davispw@gmail.com> wrote:
>>
>>> If you kill Kafka with SIGTERM, then naturally it will leave behind
>> corrupt
>>> index files. It will safely rebuild them. Don't worry.
>>>
>>> The Group Metadata Manager messages are normal -- that's just Kafka
>>> starting up.
>>>
>>> Perhaps the corrupted index log messages could be less scary, though.
>> This
>>> is a very common question from new users...
>>>
>>> -Peter
>>>
>>> On Wed, Jul 6, 2016 at 06:50 Fabian Kochem <
>>> fabian.kochem@concepts-and-training.de> wrote:
>>>
>>>> Hi,
>>>>
>>>> this is my first time using a mailing list so please forgive me if I'm
>>>> doing something wrong.
>>>>
>>>> When I start Kafka, I get a ton of messages like this:
>>>>
>>>> WARN Found a corrupted index file,
>>>> /kafka/logs/__consumer_offsets-48/00000000000000000000.index, deleting
>>>> and rebuilding index... (kafka.log.Log)
>>>> INFO Recovering unflushed segment 0 in log __consumer_offsets-48.
>>>> (kafka.log.Log)
>>>> INFO Completed load of log __consumer_offsets-48 with log end offset 0
>>>> (kafka.log.Log)
>>>> WARN Found a corrupted index file,
>>>> /kafka/logs/__consumer_offsets-19/00000000000000000000.index, deleting
>>>> and rebuilding index... (kafka.log.Log)
>>>> INFO Recovering unflushed segment 0 in log __consumer_offsets-19.
>>>> (kafka.log.Log)
>>>> INFO Completed load of log __consumer_offsets-19 with log end offset 0
>>>> (kafka.log.Log)
>>>> WARN Found a corrupted index file,
>>>> /kafka/logs/__consumer_offsets-3/00000000000000000000.index, deleting
>>>> and rebuilding index... (kafka.log.Log)
>>>> INFO Recovering unflushed segment 0 in log __consumer_offsets-3.
>>>> (kafka.log.Log)
>>>> INFO Completed load of log __consumer_offsets-3 with log end offset 0
>>>> (kafka.log.Log)
>>>>
>>>> And after that, there's an equal amount of messages which look like
>> this:
>>>>
>>>> INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
>>>> metadata from [__consumer_offsets,22]
>>>> (kafka.coordinator.GroupMetadataManager)
>>>> INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
>>>> from [__consumer_offsets,22] in 11 milliseconds.
>>>> (kafka.coordinator.GroupMetadataManager)
>>>> INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
>>>> metadata from [__consumer_offsets,25]
>>>> (kafka.coordinator.GroupMetadataManager)
>>>> INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
>>>> from [__consumer_offsets,25] in 4 milliseconds.
>>>> (kafka.coordinator.GroupMetadataManager)
>>>> INFO [Group Metadata Manager on Broker 1001]: Loading offsets and group
>>>> metadata from [__consumer_offsets,28]
>>>> (kafka.coordinator.GroupMetadataManager)
>>>> INFO [Group Metadata Manager on Broker 1001]: Finished loading offsets
>>>> from [__consumer_offsets,28] in 2 milliseconds.
>>>> (kafka.coordinator.GroupMetadataManager)
>>>>
>>>> Did I misconfigure something?
>>>>
>>>> I only have one topic with a few consumers, so this seems a bit much. I
>>>> shutdown Kafka using SIGTERM.
>>>>
>>>> Thanks in advance,
>>>> Fabian Kochem
>>>>
>>>
>>
>

-- 
Fabian Kochem
Entwicklung

---------------------------------------------
CaT Concepts and Training GmbH
Vorgebirgstraße 338
50969 Köln

Fon : +49 (0) 221 / 46 75 76 - 53
Fax : +49 (0) 221 / 46 75 76 - 09
---------------------------------------------
Geschäftsführung:

Sven Kapust, Gerald Konrad, Alexandra Oehlke,
Sandra Röbbelen, Volker Röbbelen, Denis Witt

Amtsgericht Köln HRB 57804
Steuer-Nr.: 217/5716/0769
Ust-ID-Nr.: DE 814694228
Sitz: Köln
---------------------------------------------

Mime
View raw message