kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damian Guy <damian....@gmail.com>
Subject Re: How does 'TimeWindows.of().until()' work?
Date Wed, 14 Dec 2016 09:47:41 GMT
Hi Sachin,

> windowstore.changelog.additional.retention.ms

>
> How does this relate to rentention.ms param of topic config?
> I create internal topic manually using say rentention.ms=3600000.
> In next release (post kafka_2.10-0.10.0.1) since we support delete of
> internal changelog topic as well and I want it to be retained for say just
> 1 hour.
> So how does that above parameter interfere with this topic level setting.
> Or now I just need to set above config as 3600000 and not add
> rentention.ms=3600000
> while creating internal topic.
>
>
This parameter is only used when creating the internal changelog topic for
a Window store. When the streams library creates a changelog topic for the
Window store it sets the delete policy to compact,delete. So the changelog
topic is both compacted and deleted. For the delete side of it, retention.ms
is set to the value of TimeWindows.until() + the value of
windowstore.changelog.additional.retention.ms. So if you are retaining
windows for 1 day and you just used the default for the above property then
retentions.ms=172800. Any segments older than this will be deleted from the
log.

Thanks,
Damian

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message