kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Compton <d...@danielcompton.net>
Subject Re: TTL changes - Are they retroactive?
Date Fri, 21 Nov 2014 03:25:37 GMT
Hi Parag

Just to expand on Jun’s comment, log retention and deletion is at the segment level, not
the message level. Because it’s at the segment level I would avoid using the term TTL, as
that would normally be applied to individual items.

Every log.retention.check.interval.ms (default 5 minutes), the Kafka broker will check whether
any log segment is eligible for deletion. The whole log segment must be eligible for deletion
based on the retention policy so some messages in the segment will be older than the log.retention.hours
(default 7 days) set in the Kafka config.

This isn’t a big deal normally, but is a bit different from other messaging systems so I
thought it was worth pointing out the difference.

Daniel.


> On 21/11/2014, at 12:39 pm, Jun Rao <junrao@gmail.com> wrote:
> 
> The retention configure change will be applied on existing data too.
> 
> Thanks,
> 
> Jun
> 
> On Thu, Nov 20, 2014 at 10:00 AM, Parag Shah <pshah@proofpoint.com> wrote:
> 
>> Hi all,
>> 
>>    I had a question about TTL changes to a topic’s configuration. When I
>> make the change and restart kafka, will kafka apply the changes to all
>> existing messages on disk retroactively or would it do it only for new
>> messages entering the topic?
>> 
>> Regards
>> Parag
>> 


Mime
View raw message