kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 赖剑清 <laijianq...@tp-link.com.cn>
Subject RE: retention.ms not honored for topic
Date Wed, 30 May 2018 01:22:31 GMT
Hi

I met this @kafka_v0.9.0.1, and solved by set the topic config.
You can have a try with kafka tool kafka-topic.sh and change it's config with param --config
<name=value> 

Good luck

>-----Original Message-----
>From: Thomas Hays [mailto:haystd@gmail.com]
>Sent: Wednesday, May 30, 2018 12:02 AM
>To: users@kafka.apache.org
>Subject: retention.ms not honored for topic
>
>A single topic does not appear to be honoring the retention.ms setting. Three
>other topics (plus __consumer_offsets) on the Kafka instance are deleting
>segments normally.
>
>Kafka version: 2.12-0.10.2.1
>OS: CentOS 7
>Java: openjdk version "1.8.0_161"
>Zookeeper: 3.4.6
>
>Retention settings (from kafka-topics.sh describe): Topic:elk
>PartitionCount:50 ReplicationFactor:2 Configs:retention.ms=7200000
>
>Other config settings from server.properties
>
>log.retention.hours=48
>log.segment.bytes=1073741824
>log.retention.check.interval.ms=300000
>
>Looking in the data directory, I see multiple segment files older than 48 hours:
>
>-rw-r--r-- 1 root root 1073676782 May 26 20:16 00000000004713142447.log
>-rw-r--r-- 1 root root 1073105605 May 26 20:18 00000000004715239774.log
>-rw-r--r-- 1 root root 1072907965 May 26 20:20 00000000004717450325.log
>
>Current date/time on server: Tue May 29 10:51:49 CDT 2018
>
>This issue appears on all Kafka brokers and I have tried multiple rolling restarts
>of all Kafka brokers and the issue remains. These servers stopped deleting
>segments for this topic on May 15. This does not correlate to any known
>config change. I have found no error/warning messages in the logs to indicate
>a problem.
>
>What am I missing? Thank you.
Mime
View raw message