kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Koshy <jjkosh...@gmail.com>
Subject Re: Issue with topic deletion
Date Tue, 03 Feb 2015 23:37:45 GMT
Hey Sumit,

I thought you would be providing the actual steps to reproduce :)
Nevertheless, can you get all the relevant logs: state change logs and
controller logs at the very least and if possible server logs and send
those over?

Joel

On Tue, Feb 03, 2015 at 03:27:43PM -0800, Sumit Rangwala wrote:
> In my setup kafka brokers are set for auto topic creation. In the scenario
> below a node informs other nodes (currently 5 in total) about  a number of
> new (non-existent) topics, and  all the nodes almost simultaneously open a
> consumer for each of those topics. Sometime later another node informs all
> other nodes of a new list of topics and each node, if they find that an
> older topic exists in kafka, goes ahead and deletes the older topic. What I
> have found is that many of the topics stay in the "marked for deletion"
> state forever.
> 
> 
> I get the list of topics using ZkUtils.getAllTopics(zkClient) and delete
> topics using AdminUtils.deleteTopic(zkClient, topic). Since many nodes
> might try to delete the same topic at the same time I do
> see ZkNodeExistsException while deleting the topic, which I catch an
> ignore. (e.g., org.apache.zookeeper.KeeperException$NodeExistsException:
> KeeperErrorCode = NodeExists for
> /admin/delete_topics/LAX1-GRIFFIN-r13-1423001701601)
> 
> # State of one deleted topic on kafka brokers:
> Topic:LAX1-GRIFFIN-r13-1423001701601 PartitionCount:8 ReplicationFactor:1
> Configs:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 0 Leader: -1 Replicas:
> 49558 Isr:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 1 Leader: -1 Replicas:
> 49554 Isr:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 2 Leader: -1 Replicas:
> 49557 Isr:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 3 Leader: -1 Replicas:
> 49558 Isr:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 4 Leader: -1 Replicas:
> 49554 Isr:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 5 Leader: -1 Replicas:
> 49557 Isr:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 6 Leader: -1 Replicas:
> 49558 Isr:
> Topic: LAX1-GRIFFIN-r13-1423001701601 Partition: 7 Leader: -1 Replicas:
> 49554 Isr:
> 
> 
> # Controller log says
> 
> [2015-02-03 22:59:03,399] INFO [delete-topics-thread-49554], Deletion for
> replicas 49557,49554,49558 for partition
> [LAX1-GRIFFIN-r13-1423001701601,0],[LAX1-GRIFFIN-r13-1423001701601,6],[LAX1-GRIFFIN-r13-1423001701601,5],[LAX1-GRIFFIN-r13-1423001701601,3],[LAX1-GRIFFIN-r13-1423001701601,7],[LAX1-GRIFFIN-r13-1423001701601,1],[LAX1-GRIFFIN-r13-1423001701601,4],[LAX1-GRIFFIN-r13-1423001701601,2]
> of topic LAX1-GRIFFIN-r13-1423001701601 in progress
> (kafka.controller.TopicDeletionManager$DeleteTopicsThread)
> 
> current time: Tue Feb  3 23:20:58 UTC 2015
> 
> 
> Since I don't know the delete topic algorithm, I am not sure why sure these
> topics are not garbage collected. I do have the complete setup running in
> docker right now on my local box so please let me know if any more info is
> required to troubleshoot this issue.
> 
> 
> Furthermore, does ZkUtils.getAllTopics(zkClient) return "marked for
> deletion" topic as well. If so, is there a easy way to get a list of active
> topics (other than looking at all the topics in /admin/delete_topics/ and
> taking a set difference with the topics returned
> by ZkUtils.getAllTopics(zkClient) )
> 
> Sumit
> (More setup info below)
> 
> 
> Setup
> --------
> Zookeeper: 3.4.6
> Kafka broker: 0.8.2-beta
> Kafka clients: 0.8.2-beta
> 
> # Kafka boker settings (all other settings are default 0.8.2-beta settings)
> kafka.controlled.shutdown.enable: 'FALSE'
> kafka.auto.create.topics.enable: 'TRUE'
> kafka.num.partitions: 8
> kafka.default.replication.factor: 1
> kafka.rebalance.backoff.ms: 3000
> kafka.rebalance.max.retries: 10
> kafka.log.retention.minutes: 1200
> kafka.delete.topic.enable: 'TRUE'


Mime
View raw message