kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian McCague <Adrian.McCa...@zopa.com>
Subject RE: Topic deletion
Date Sun, 09 Apr 2017 16:37:26 GMT
Thank you Mayuresh, that's an excellent write-up.

As you mention about reassignment blocking deletion; I think we are seeing an issue with our
brokers and reassignment so that could well be the root cause of the deletion issue we experienced.
The zKclient suggestion that I followed before resolved the issue on ZK but the brokers never
resolved so we rebuilt the system (luckily this is in a staging environment).

Thanks
Adrian

-----Original Message-----
From: Mayuresh Gharat [mailto:gharatmayuresh15@gmail.com] 
Sent: 07 April 2017 18:56
To: users@kafka.apache.org
Subject: Re: Topic deletion

Hi Adrian,

When you delete a topic, it is marked a under /admin/delete_topics path in zookeeper. Once
that is done, controller gets a notification that a topic is marked for delete. The controller
then notifies the replcias of the topic-partitions to delete the data for the topic and the
topic itself.
After KAFKA-1911, delete topic is made asynchronous and faster. The replicas on receiving
the notification from controller will rename the topic-partition directory to something like
topic-partition.delete and return to controller saying delete topic is done. The controller
when it receives the ack from all the replicas that topic has been deleted, it will delete
the znode under /admin/delete_topics path. Your delete topic is successful at this stage.
The renamed directory gets removed asynchronously on those replicas brokers at some later
point in time.
Also remember, if you topic-partitions are been reassigned, delete topic will not be triggered
till the reassignment is finished.

We at Linkedin, have been suing Delete topic for a while now and it has been working fine
after KAFKA-1911.

Hope this helps.

Thanks,

Mayuresh

On Fri, Apr 7, 2017 at 2:15 AM, Adrian McCague <Adrian.McCague@zopa.com>
wrote:

> Indeed, I wish it was explained somewhere what marked for deletion 
> does, and what process handles it. I will have to go and investigate the source.
>
> I can confirm the zkCli did the trick, thanks for the hint!
>
> Adrian
>
> -----Original Message-----
> From: Akhilesh Pathodia [mailto:pathodia.akhilesh@gmail.com]
> Sent: 07 April 2017 09:57
> To: users@kafka.apache.org
> Subject: Re: Topic deletion
>
> I am not sure but kafka delete command does not delete the topic 
> actually, it only marks it for deletion. Probably it is fixed in later 
> version of kafka.
>
> On Fri, Apr 7, 2017 at 2:14 PM, Adrian McCague 
> <Adrian.McCague@zopa.com>
> wrote:
>
> > Hi Akhilesh,
> >
> > Why would this approach need to be taken over the kafka-topics tool, 
> > out of interest?
> >
> > Thanks
> > Adrian
> >
> > -----Original Message-----
> > From: Akhilesh Pathodia [mailto:pathodia.akhilesh@gmail.com]
> > Sent: 07 April 2017 09:37
> > To: users@kafka.apache.org
> > Subject: Re: Topic deletion
> >
> > Hi Adrian,
> >
> > You will have to delete the broker directory from zookeeper. This 
> > can be done  from zookeeper cli. Connect to zookeeper cli using 
> > below
> command:
> >
> > zookeeper-client -server <ZOOKEEPER_URI>
> >
> > Then run below command :
> >
> > rmr /brokers/topics/<TOPIC_NAME>
> >
> > Thanks,
> > AKhilesh
> >
> > On Thu, Apr 6, 2017 at 11:03 PM, Adrian McCague 
> > <Adrian.McCague@zopa.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I am trying to understand topic deletion in kafka, there appears 
> > > to be very little documentation or answers on how this works. 
> > > Typically they just say to turn on the feature on the broker (in my case it
is).
> > >
> > > I executed:
> > > Kafka-topics.bat -delete -zookeeper keeperhere -topic mytopic
> > >
> > > Running this again yields:
> > > Topic mytopic is already marked for deletion.
> > >
> > > --describe yields:
> > > Topic: mytopic  PartitionCount:6        ReplicationFactor:3
>  Configs:
> > > retention.ms=0
> > >         Topic: mytopic  Partition: 0    Leader: -1      Replicas:
> > > 1006,1001,1005        Isr:
> > >         Topic  mytopic  Partition: 1    Leader: -1      Replicas:
> > > 1001,1005,1003        Isr:
> > >        Topic: mytopic  Partition: 2    Leader: -1      Replicas:
> > > 1005,1003,1004        Isr:
> > >         Topic: mytopic  Partition: 3    Leader: -1      Replicas:
> > > 1003,1004,1007        Isr:
> > >         Topic: mytopic  Partition: 4    Leader: -1      Replicas:
> > > 1004,1007,1006        Isr:
> > >         Topic: mytopic  Partition: 5    Leader: -1      Replicas:
> > > 1007,1006,1001        Isr:
> > >
> > > You can see that the deletion mark has meant that the Leader is -1.
> > > Also I read somewhere that retention needs to be set to something 
> > > low to trigger the deletion, hence the config of retention.ms=0
> > >
> > > Consumers (or streams in my case) no longer see the topic:
> > > org.apache.kafka.streams.errors.TopologyBuilderException: Invalid 
> > > topology building: stream-thread [StreamThread-1] Topic not found:
> > > mytopic
> > >
> > > And I can't create a new topic in its place:
> > > [2017-04-06 18:26:00,702] ERROR org.apache.kafka.common.
> > errors.TopicExistsException:
> > > Topic 'mytopic' already exists. (kafka.admin.TopicCommand$)
> > >
> > > I am a little lost as to where to go next, could someone explain 
> > > how topic deletion is actually applied when a topic is 'marked' 
> > > for deletion as that may help trigger it.
> > >
> > > Thanks!
> > > Adrian
> > >
> > >
> >
>



--
-Regards,
Mayuresh R. Gharat
(862) 250-7125
Mime
View raw message