kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jagbir Hooda <jho...@gmail.com>
Subject Re: How to delete defunct topics
Date Fri, 06 Feb 2015 22:37:46 GMT
Hi Joel,

The mbean approach depends upon storing traffic stats over a period of
time, so is doable but complex. I am thinking about another approach
to check logfiles time-stamp (to check producer activity) in
conjunction with the mtime for topic offsets (consumers activity). So
if both timestamp and mtime are older than 'n' days that may imply a
defunct topic. Will appreciate your feedback.

Thanks,
Jagbir

On Thu, Feb 5, 2015 at 6:57 PM, Joel Koshy <jjkoshy.w@gmail.com> wrote:
> There are mbeans
> (http://kafka.apache.org/documentation.html#monitoring) that you can
> poke for incoming message rate - if you look at those over a period of
> time you can figure out which of those are likely to be defunct and
> then delete those topics.
>
> On Thu, Feb 05, 2015 at 02:38:27PM -0800, Jagbir Hooda wrote:
>> First I would like to take this opportunity to thank this group for
>> releasing 0.8.2.0. It's a major milestone with a rich set of features.
>> Kudos to all the contributors! We are still running 0.8.1.2 and are
>> planning to upgrade to 0.8.2.0. While planning this upgrade we
>> discovered many topics that are no longer active and best be deleted.
>> Since it would be a common task faced by many kafka adopters I thought
>> I'd raise it here and seek expert advise. Basically what we desire is
>> a utility/script/steps to first identify the defunct topics (let's say
>> those topics that haven't seen any traffic in the past 'n' days) and
>> then delete them. Will appreciate your response.
>>
>> Thanks,
>> Jagbir
>

Mime
View raw message