kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Holland <john.holl...@objectpartners.com>
Subject Re: Verify log compaction
Date Sat, 30 Jul 2016 01:20:23 GMT
Check the log-cleaner.log file on the server.  When the thread runs you'll
see output for every partition it compacts and the compaction ratio it
achieved.

The __consumer_offsets topic is compacted, I see log output from it being
compacted frequently.

Depending on your settings for the topic it may take a while for it to
compact.  Compaction doesn't occur on the current log segment.  Look at
these settings for the topic, "segment.bytes" and "segment.ms".  Lower them
to force quicker compaction.

On 0.8.2.2, occasionally, the compaction thread would die and then the
__consumer_offets topic would grow out of control.  Kafka logs the thread
death in the log-cleaner.log.


On Fri, Jul 29, 2016 at 4:10 PM David Yu <guanshan@gmail.com> wrote:

> Hi,
>
> We are using Kafka 0.9.0.0. One of our topic is set to use log compaction.
> We have also set log.cleaner.enable. However, we suspected that the topic
> is not being compacted.
>
> What is the best way for us to verify the compaction is happening?
>
> Thanks,
> David
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message