kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krzysztof Zarzycki <k.zarzy...@gmail.com>
Subject retention.ms is not respected after upgrade to 0.8.2
Date Mon, 29 Jun 2015 06:36:48 GMT
Greetings!
I have problem with Kafka. I had a cluster of 3 brokers in version 0.8.1. I
have a very important topic with raw events, that had a config
retention.ms={365
days in ms} .
It all worked, fine, data was not being deleted.
But now I upgraded all brokers to 0.8.2 and suddenly  brokers delete the
data! They don't respect retention.ms.
I have no other settings around retention set: global log.retention.hours
is set to default 168, log.retention.bytes is not set.

Some more info:
1. I tried looking into ZK config and it looks fine:
$ get /kafka081/config/topics/my_topic
{"version":1,"config":{"retention.ms":"31536000000"}}
cZxid = 0xa0006412c
ctime = Tue Mar 31 15:02:20 CEST 2015
mZxid = 0x1000016bb7
mtime = Fri Jun 26 22:28:40 CEST 2015
pZxid = 0xa0006412c
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 53
numChildren = 0

2. I tried to overwrite retention.ms once again for my topic. Didn't help.

3. I looked into logs of the broker and found that it indeed deletes the
data, but it doesn't print *why* (based on what rule) it deletes the data:
[2015-06-29 07:35:40,861] INFO Deleting segment 89226232 from log
my_topic-1. (kafka.log.Log)
[2015-06-29 07:35:40,993] INFO Deleting index
/var/lib/kafka/kafka-logs-1/my_topic-1/00000000000089226232.index.deleted (k
afka.log.OffsetIndex)


Please help me, I have no idea what to do about it. Any hint on at least
how to debug a problem would be great!

Cheers,
Krzysztof Zarzycki

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message