kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Yadav <thekarthikya...@gmail.com>
Subject Method to check if the log-cleaner of a Kafka broker is running or not
Date Fri, 12 Apr 2019 08:03:53 GMT
I have a few brokers which take care of the data of a few applications. I
have the log-cleaner for all the brokers enabled with thecleanup.policy as
delete. There was an issue a few weeks back where the log-cleaner stopped
working in one of the brokers and this led complete disk usage, and it
wasn't detectable.

I am looking for an efficient way to *check if the log-cleaner of a broker
has stopped working*.

Firstly, I am using the *Kafka 1.0.2 version*.

As a simple method, I have tried looking into the logs of the log-cleaner,
and when the log wouldn't update for a considerable amount of time meant
the log-cleaner stopped working.

But some of the brokers' log-cleaner's log was not getting updated for some
reason even though the log-cleaner was running(understood because the size
of the log of the applications didn't keep increasing after a considerable
amount of time).

I have tried playing around with a few configs,
log.cleaner.min.cleanable.ratiolog.retention.check.interval.ms etc. on why
the log-cleaner's log was not getting updated, but I couldn't conclude on
anything. *Any ideas on how to check if the log-cleaner is running or why
the log-cleaner's log wouldn't log?*

TLDR;

Had some trouble as the log-cleaner of a Kafka broker stopped and couldn't
detect it in time. Looking for an efficient method to check the termination
of the log-cleaner.
ᐧ

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message