kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: Effect of settings segment.ms and retention.ms not accurate
Date Tue, 29 May 2018 06:07:23 GMT
Retention time is a lower bound for how long it is guaranteed that data
will be stored. This guarantee work "one way" only. There is no
guarantee when data will be deleted after the bound passed.

However, client side, you can always check the record timestamp and just
drop older data that is still in the topic.

Hope this helps.


-Matthias


On 5/28/18 9:18 PM, Shantanu Deshmukh wrote:
> Please help.
> 
> On Mon, May 28, 2018 at 5:18 PM Shantanu Deshmukh <shantanu88d@gmail.com>
> wrote:
> 
>>  I have a topic otp-sms. I want that retention of this topic should be 5
>> minutes as OTPs are invalid post that amount of time. So I set
>> retention.ms=300000. However, this was not working. So reading more in
>> depth in Kafka configuration document I found another topic level setting
>> that can be tuned for topic retention to work properly. So I set
>> segment.ms=300000 as well.
>>
>> After these changes I saw that old logs go deleted. Still in the topic I
>> could see one record which is more than 15 minutes old and not getting
>> deleted. What does one have to do to actually delete messages generated n
>> minutes ago?
>>
> 


Mime
View raw message