kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rosenberg <...@squareup.com>
Subject Re: log.retention.bytes.per.topic does not trigger deletion
Date Mon, 20 Jan 2014 05:19:35 GMT
Please be sure to update the online config docs with this change!  The
per topic options are still listed there....

Jason

On Thu, Jan 16, 2014 at 9:57 PM, Ben Summer <bsummer@gnipcentral.com> wrote:
> I see. I don't have version 0.8.1 yet. We just updated to 0.8.0 from beta
> after it became the "stable version".
> Good to know there is a fix for this. I'll start trying it out in some
> non-production environments.
>
> Thanks,
> Ben
>
>
> On Thu, Jan 16, 2014 at 7:42 PM, Guozhang Wang <wangguoz@gmail.com> wrote:
>
>> In the latest version, per-topic configs have been moved to Zookeeper you
>> and set them using admin tools instead of writing the config files. Could
>> you try trunk HEAD and see if this issue has already been resolved:
>>
>> https://issues.apache.org/jira/browse/KAFKA-554
>>
>> Guozhang
>>
>>
>> On Thu, Jan 16, 2014 at 6:27 PM, Ben Summer <bsummer@gnipcentral.com>
>> wrote:
>>
>> > v0.8.0, (non-beta) downloaded from the website. Let me know if there is a
>> > file I can check in the installation footprint that can give you an
>> actual
>> > change id.
>> >
>> > Is there a fix to this particular feature that you know was made in a
>> later
>> > build?
>> >
>> > Thanks for the quick response,
>> > Ben
>> >
>> >
>> > On Thu, Jan 16, 2014 at 6:22 PM, Guozhang Wang <wangguoz@gmail.com>
>> wrote:
>> >
>> > > Hello Ben,
>> > >
>> > > Which version are you using?
>> > >
>> > > Guozhang
>> > >
>> > >
>> > > On Thu, Jan 16, 2014 at 3:15 PM, Ben Summer <bsummer@gnipcentral.com>
>> > > wrote:
>> > >
>> > > > I tried using the following two retention properties
>> > > >
>> > > > log.retention.bytes=3221225472
>> > > >
>> > > >
>> > >
>> >
>> log.retention.bytes.per.topic=first_topic:1099511627776,second_topic:1099511627776
>> > > >
>> > > > which I interpret to mean "by default, keep 3GB per topic partition,
>> > > except
>> > > > for first_topic and second_topic, which should retain 1TB each
>> (across
>> > > all
>> > > > their partitions).
>> > > >
>> > > > I've noticed that the data for first_topic and second_topic grow way
>> > > beyond
>> > > > 1TB and no deletions occur. Is the syntax correct?
>> > > >
>> > > > Thanks in advance,
>> > > > Ben
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > -- Guozhang
>> > >
>> >
>>
>>
>>
>> --
>> -- Guozhang
>>

Mime
View raw message