kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Visher <tim.vis...@gmail.com>
Subject Re: Deleting a topic without delete.topic.enable=true?
Date Mon, 12 Dec 2016 16:47:21 GMT
I wonder if datadog monitoring triggers that behavior. That's the only
other piece of our infrastructure that may have been talking to that topic.

On Mon, Dec 12, 2016 at 12:40 AM, Surendra , Manchikanti <
surendra.manchikanti@gmail.com> wrote:

> If "auto.create.topics.enable" is set to true in your configurations , any
> producer/consumer or fetch request will create the topic again. Set it to
> false and delete the topic.
>
> -- Surendra Manchikanti
>
> On Sat, Dec 10, 2016 at 10:59 AM, Todd Palino <tpalino@gmail.com> wrote:
>
> > Are you running something else besides the consumers that would maintain
> a
> > memory of the topics and potentially recreate them by issuing a metadata
> > request? For example, Burrow (the consumer monitoring app I wrote) does
> > this because it maintains a list of all topics in memory, and will end up
> > recreating a topic that has been deleted as it issues a metadata request
> to
> > try and find out what happened after an offset request for the topic
> fails.
> >
> > -Todd
> >
> >
> > On Fri, Dec 9, 2016 at 8:37 AM, Tim Visher <tim.visher@gmail.com> wrote:
> >
> > > On Fri, Dec 9, 2016 at 11:34 AM, Todd Palino <tpalino@gmail.com>
> wrote:
> > >
> > > > Given that you said you removed the log directories, and provided
> that
> > > when
> > > > you did the rmr on Zookeeper it was to the “/brokers/topics/(topic
> > name)”
> > > > path, you did the right things for a manual deletion. It sounds like
> > you
> > > > may have a consumer (or other client) that is recreating the topic.
> Do
> > > you
> > > > have auto topic creation enabled?
> > > >
> > >
> > > That was the last epiphany we had. We had shut down the producer but
> not
> > > all the consumers and we do allow auto-topic creation.
> > >
> > > That said, we then proceeded to shut all of them down (the consumers)
> and
> > > the topic came back. I'm glad that we were doing the right steps
> though.
> > >
> > > >
> > > > -Todd
> > > >
> > > >
> > > > On Fri, Dec 9, 2016 at 8:25 AM, Tim Visher <tim.visher@gmail.com>
> > wrote:
> > > >
> > > > > I did all of that because setting delete.topic.enable=true wasn't
> > > > > effective. We set that across every broker, restarted them, and
> then
> > > > > deleted the topic, and it was still stuck in existence.
> > > > >
> > > > > On Fri, Dec 9, 2016 at 11:11 AM, Ali Akhtar <ali.rac200@gmail.com>
> > > > wrote:
> > > > >
> > > > > > You need to also delete / restart zookeeper, its probably storing
> > the
> > > > > > topics there. (Or yeah, just enable it and then delete the topic)
> > > > > >
> > > > > > On Fri, Dec 9, 2016 at 9:09 PM, Rodrigo Sandoval <
> > > > > > rodrigo.madferit@gmail.com
> > > > > > > wrote:
> > > > > >
> > > > > > > Why did you do all those things instead of just setting
> > > > > > > delete.topic.enable=true?
> > > > > > >
> > > > > > > On Dec 9, 2016 13:40, "Tim Visher" <tim.visher@gmail.com>
> wrote:
> > > > > > >
> > > > > > > > Hi Everyone,
> > > > > > > >
> > > > > > > > I'm really confused at the moment. We created a topic
with
> > > brokers
> > > > > set
> > > > > > to
> > > > > > > > delete.topic.enable=false.
> > > > > > > >
> > > > > > > > We now need to delete that topic. To do that we shut
down all
> > the
> > > > > > > brokers,
> > > > > > > > deleted everything under log.dirs and logs.dir on
all the
> kafka
> > > > > > brokers,
> > > > > > > > `rmr`ed the entire chroot that kafka was storing things
under
> > in
> > > > > > > zookeeper,
> > > > > > > > and then brought kafka back up.
> > > > > > > >
> > > > > > > > After doing all that, the topic comes back, every
time.
> > > > > > > >
> > > > > > > > What can we do to delete that topic?
> > > > > > > >
> > > > > > > > --
> > > > > > > >
> > > > > > > > In Christ,
> > > > > > > >
> > > > > > > > Timmy V.
> > > > > > > >
> > > > > > > > http://blog.twonegatives.com/
> > > > > > > > http://five.sentenc.es/ -- Spend less time on mail
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > *Todd Palino*
> > > > Staff Site Reliability Engineer
> > > > Data Infrastructure Streaming
> > > >
> > > >
> > > >
> > > > linkedin.com/in/toddpalino
> > > >
> > >
> >
> >
> >
> > --
> > *Todd Palino*
> > Staff Site Reliability Engineer
> > Data Infrastructure Streaming
> >
> >
> >
> > linkedin.com/in/toddpalino
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message