kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Rangwala <sumitrangw...@gmail.com>
Subject Re: unable to delete topic with 0.8.2 rc2
Date Mon, 26 Jan 2015 22:02:36 GMT
I was planning to start a new thread by my experience is similar to the one
described here. I am currently using 0.8.2-beta and my setup has a producer
producing to topic "DC-DATA", kafka brokers with auto-topic-creation
enabled, and mirrormaker set to replicate any ".*-DATA" topics. After the
producer produces some data another thread in my code goes ahead and
deletes the topic DC-DATA. And I am seeing a similar behavior, i.e, the
topic reappears after deleting. I, however, changed the thread in my code
to wake up periodically and if it finds the topic (using getAllTopics) it
tries to delete the topic. What I found interesting is that after few
delete attempts the topic indeed gets deleted and does not reappear.  Seems
like some interplay between auto-topic creating and consumer notification
when a consumer is subscribed to a topic specified as a regex.


Sumit





On Mon, Jan 26, 2015 at 1:49 PM, Jun Rao <jun@confluent.io> wrote:

> Joel,
>
> That's probably because console consumer always uses wildcard for
> consumption.
>
> Thanks,
>
> Jun
>
> On Mon, Jan 26, 2015 at 1:44 PM, Joel Koshy <jjkoshy.w@gmail.com> wrote:
>
> > I think this should work even on non-wildcard consumers. (i.e., I just
> > tried it locally and it appears to work fine).
> >
> > Here is what I did:
> >
> > - Leave a console consumer up and running, bounce the broker to turn
> >   off auto-create
> > - Delete the topic
> > - The consumer should rebalance and stop consuming that topic
> > - Bounce the broker to turn on auto-create
> >
> > The only issue is that while your broker has auto-create off you will
> > see errors on some producers - which you can announce as a brief
> > outage to your users.
> >
> > Joel
> >
> > On Mon, Jan 26, 2015 at 01:02:57PM -0800, Jun Rao wrote:
> > > Actually, I think the deletion of a topic will trigger a rebalance on
> all
> > > wildcard subscribers. The time to complete the rebalance depends on
> the #
> > > of topic/partitions.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Jan 26, 2015 at 12:26 PM, Jason Rosenberg <jbr@squareup.com>
> > wrote:
> > >
> > > > Yeah Joel,
> > > >
> > > > I just thought of that idea too (e.g. temporarily disable auto topic
> > > > creation).  I can probably make that work (it may result in
> unexpected
> > lack
> > > > of topic creation, but I can message this as a maintenance downtime,
> > etc.).
> > > >
> > > > So, that was my next question, so a consumer rebalance will trigger a
> > fresh
> > > > topic pull from the consumers?  How long is 'long enough' to ensure a
> > > > rebalance has occurred everywhere?
> > > >
> > > > Jason
> > > >
> > > > On Mon, Jan 26, 2015 at 3:07 PM, Joel Koshy <jjkoshy.w@gmail.com>
> > wrote:
> > > >
> > > > > Hey Jason,
> > > > >
> > > > > Is it an option for you to do the following:
> > > > >
> > > > > - Bounce in a config change to the brokers to turn off auto-create
> > > > > - (Batch)-delete the topic(s)
> > > > > - Wait long enough for consumers to rebalance (after which they
> will
> > > > >   no longer consume the topic(s))
> > > > > - Bounce in a config change to the brokers to turn on auto-create
> > > > >
> > > > > Joel
> > > > >
> > > > > On Mon, Jan 26, 2015 at 12:01:59PM -0800, Jun Rao wrote:
> > > > > > Hi, Jason,
> > > > > >
> > > > > > I am not sure that we should delay the 0.8.2 release. The reasons
> > are
> > > > (1)
> > > > > > There are other features such as the new java producer and
> > Kafka-based
> > > > > > offset management that are potentially useful to people. (2) It
> may
> > > > take
> > > > > > some time to completely fix the issue with deleting topic since
> it
> > > > > involves
> > > > > > wire protocol changes.
> > > > > >
> > > > > > What we can do is probably just to document the limitation of
> > delete
> > > > > topic
> > > > > > in 0.8.2.0 in our site.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Mon, Jan 26, 2015 at 11:05 AM, Jason Rosenberg <
> > jbr@squareup.com>
> > > > > wrote:
> > > > > >
> > > > > > > I think this shortcoming should probably delay release of 0.8.2
> > until
> > > > > > > resolved, no?
> > > > > > >
> > > > > > > On Mon, Jan 26, 2015 at 2:01 PM, Guozhang Wang <
> > wangguoz@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > It then seems to me that delete-topic will not actually work
> > > > > "smoothly"
> > > > > > > > until create topic request is added since it is too much to
> > require
> > > > > > > people
> > > > > > > > to turn off their clients while deleting topics. In this case
> > shall
> > > > > we
> > > > > > > make
> > > > > > > > it clear in the release docs of in 0.8.2, or even still mark
> > it as
> > > > > > > > not-supported?
> > > > > > > >
> > > > > > > > Guozhang
> > > > > > > >
> > > > > > > > On Mon, Jan 26, 2015 at 8:14 AM, Harsha <kafka@harsha.io>
> > wrote:
> > > > > > > >
> > > > > > > > > Jun,
> > > > > > > > >       I made an attempt at fixing that issue as part of
> this
> > JIRA
> > > > > > > > >       https://issues.apache.org/jira/browse/KAFKA-1507 .
> > > > > > > > > As Jay pointed out there should be admin api if there is
> more
> > > > info
> > > > > on
> > > > > > > > > this api I am interested in adding/fixing this issue.
> > > > > > > > > Thanks,
> > > > > > > > > Harsha
> > > > > > > > >
> > > > > > > > > On Mon, Jan 26, 2015, at 07:28 AM, Jun Rao wrote:
> > > > > > > > > > Yes, that's the issue. Currently, topics can be
> > auto-created on
> > > > > > > > > > TopicMetadataRequest, which can be issued from both the
> > > > producer
> > > > > and
> > > > > > > > the
> > > > > > > > > > consumer. To prevent that, you would need to stop the
> > producer
> > > > > and
> > > > > > > the
> > > > > > > > > > consumer before deleting a topic. We plan to address this
> > issue
> > > > > once
> > > > > > > we
> > > > > > > > > > have a separate request for creating topics.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > On Mon, Jan 26, 2015 at 7:21 AM, Harsha <kafka@harsha.io
> >
> > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > There could be another case where if you have
> > > > > > > > auto.create.topics.enable
> > > > > > > > > > > to set to true ( its true by default) . Any
> > > > > TopicMetadataRequest
> > > > > > > can
> > > > > > > > > > > recreate topics. So if you issued a delete topic
> command
> > and
> > > > > you
> > > > > > > have
> > > > > > > > > > > producers running or consumers? too which is issuing a
> > > > > > > > > > > TopicMetadataRequest than the topic will be recreated.
> > > > > > > > > > > -Harsha
> > > > > > > > > > >
> > > > > > > > > > > On Sun, Jan 25, 2015, at 11:26 PM, Jason Rosenberg
> wrote:
> > > > > > > > > > > > cversion did change (incremented by 2) when I issue
> the
> > > > > delete
> > > > > > > > > command.
> > > > > > > > > > > >
> > > > > > > > > > > > From the logs on the conroller broker (also the
> leader
> > for
> > > > > the
> > > > > > > > > topic), it
> > > > > > > > > > > > looks like the delete proceeds, and then the topic
> gets
> > > > > recreated
> > > > > > > > > > > > immediately (highlighted in yellow). It appears maybe
> > it’s
> > > > > due
> > > > > > > to a
> > > > > > > > > > > > consumer client app trying to consume the topic.
> Also,
> > the
> > > > > > > consumer
> > > > > > > > > is
> > > > > > > > > > > > not
> > > > > > > > > > > > yet updated to 0.8.2 (it’s using 0.8.1.1), perhaps
> > that’s
> > > > > part of
> > > > > > > > the
> > > > > > > > > > > > problem?
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > 2015-01-26 07:02:14,281  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > >
> controller.PartitionStateMachine$DeleteTopicsListener -
> > > > > > > > > > > > [DeleteTopicsListener on 6]: Starting topic deletion
> > for
> > > > > topics
> > > > > > > > > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,282  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Handling deletion for
> topics
> > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,286  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Deletion of topic mytopic
> > > > > (re)started
> > > > > > > > > > > > 2015-01-26 07:02:14,286  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager - [Topic Deletion
> > Manager
> > > > 6],
> > > > > > > Topic
> > > > > > > > > > > > deletion callback for mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,289  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager - [Topic Deletion
> > Manager
> > > > 6],
> > > > > > > > > > > > Partition deletion callback for [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,295  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to OfflineReplica for
> > replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
> [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,303  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New
> > leader and
> > > > > ISR
> > > > > > > for
> > > > > > > > > > > > partition [mytopic,0] is
> > > > > {"leader":6,"leader_epoch":1,"isr":[6]}
> > > > > > > > > > > > 2015-01-26 07:02:14,312  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New
> > leader and
> > > > > ISR
> > > > > > > for
> > > > > > > > > > > > partition [mytopic,0] is
> > > > > {"leader":-1,"leader_epoch":2,"isr":[]}
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to ReplicaDeletionStarted
> for
> > > > > replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
> [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [kafka-request-handler-5]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [kafka-request-handler-7]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [kafka-request-handler-7]
> > > > > > > > > > > > log.OffsetIndex - Deleting index
> > > > > > > > > > > > /mypath/mytopic-0/00000000000000000000.index
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [kafka-request-handler-7]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Deleted log for partition [mytopic,0] in
> > > > /mypath/mytopic-0.
> > > > > > > > > > > > 2015-01-26 07:02:14,314  INFO
> > > > > > > > [Controller-6-to-broker-6-send-thread]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to
> ReplicaDeletionSuccessful
> > for
> > > > > > > replicas
> > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,314  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Handling deletion for
> topics
> > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,316  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Deletion for replicas 7 for
> > > > > partition
> > > > > > > > > > > > [mytopic,0] of topic mytopic in progress
> > > > > > > > > > > > 2015-01-26 07:02:14,316  INFO
> > > > > > > > [Controller-6-to-broker-7-send-thread]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to
> ReplicaDeletionSuccessful
> > for
> > > > > > > replicas
> > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7]
> > > > > > > > > > > > 2015-01-26 07:02:14,316  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Handling deletion for
> topics
> > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,318  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to NonExistentReplica for
> > > > replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
> [Topic=mytopic,Partition=0,Replica=6],[Topic=mytopic,Partition=0,Replica=7]
> > > > > > > > > > > > 2015-01-26 07:02:14,318  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to
> > OfflinePartition
> > > > for
> > > > > > > > > > > > partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,318  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to
> > > > NonExistentPartition
> > > > > for
> > > > > > > > > > > > partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,325  WARN
> [kafka-request-handler-2]
> > > > > > > > > > > > server.ReplicaManager - [Replica Manager on Broker
> 6]:
> > > > Fetch
> > > > > > > > request
> > > > > > > > > > > > with correlation id 2619442 from client
> > > > > ReplicaFetcherThread-0-6
> > > > > > > on
> > > > > > > > > > > > partition [mytopic,0] failed due to Partition
> > [mytopic,0]
> > > > > doesn't
> > > > > > > > > > > > exist on 6
> > > > > > > > > > > > 2015-01-26 07:02:14,371  INFO
> [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Deletion of topic mytopic
> > > > > successfully
> > > > > > > > > > > > completed
> > > > > > > > > > > > 2015-01-26 07:02:14,376  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine$TopicChangeListener
> -
> > > > > > > > > > > > [TopicChangeListener on Controller 6]: New topics:
> > [Set()],
> > > > > > > deleted
> > > > > > > > > > > > topics: [Set()], new partition replica assignment
> > [Map()]
> > > > > > > > > > > > 2015-01-26 07:02:14,403  WARN
> [request-expiration-task]
> > > > > > > > > > > > server.ReplicaManager - [Replica Manager on Broker
> 6]:
> > > > Fetch
> > > > > > > > request
> > > > > > > > > > > > with correlation id 2054626 from client
> > > > > > > > > > > > myconsumer-1415333962444-cc78e67e-0-6 on partition
> > > > > [mytopic,0]
> > > > > > > > failed
> > > > > > > > > > > > due to Partition [mytopic,0] doesn't exist on 6
> > > > > > > > > > > > 2015-01-26 07:02:14,446  INFO
> [kafka-request-handler-5]
> > > > > > > > > > > > admin.AdminUtils$ - Topic creation
> > > > > > > > > > > > {"version":1,"partitions":{"0":[7,6]}}
> > > > > > > > > > > > 2015-01-26 07:02:14,457  INFO
> [kafka-request-handler-5]
> > > > > > > > > > > > server.KafkaApis - [KafkaApi-6] Auto creation of
> topic
> > > > > mytopic
> > > > > > > > with 1
> > > > > > > > > > > > partitions and replication factor 2 is successful!
> > > > > > > > > > > > 2015-01-26 07:02:14,458  INFO
> > > > [kafka-network-thread-27330-2]
> > > > > > > > > > > > network.Processor - Closing socket connection to /
> > > > 10.1.10.4.
> > > > > > > > > > > > 2015-01-26 07:02:14,466  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine$TopicChangeListener
> -
> > > > > > > > > > > > [TopicChangeListener on Controller 6]: New topics:
> > > > > > > [Set(mytopic)],
> > > > > > > > > > > > deleted topics: [Set()], new partition replica
> > assignment
> > > > > > > > > > > > [Map([mytopic,0] -> List(7, 6))]
> > > > > > > > > > > > 2015-01-26 07:02:14,466  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New
> topic
> > > > > creation
> > > > > > > > > > > > callback for [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,466  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New
> > partition
> > > > > > > creation
> > > > > > > > > > > > callback for [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,466  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to NewPartition
> > for
> > > > > > > partitions
> > > > > > > > > > > > [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,467  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to NewReplica for replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
> [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,468  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to
> > OnlinePartition for
> > > > > > > > > partitions
> > > > > > > > > > > > [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,530  INFO
> > > > > > > > > > > >
> [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to OnlineReplica for
> replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> >
> [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,532  INFO
> [kafka-request-handler-5]
> > > > > log.Log -
> > > > > > > > > > > > Completed load of log mytopic-0 with log end offset 0
> > > > > > > > > > > > 2015-01-26 07:02:14,533  INFO
> [kafka-request-handler-5]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Created log for partition [mytopic,0] in /mypath
> with
> > > > > > > properties
> > > > > > > > > > > > {segment.index.bytes -> 10485760,
> file.delete.delay.ms
> > ->
> > > > > 60000,
> > > > > > > > > > > > segment.bytes -> 1073741824, flush.ms ->
> > > > > 9223372036854775807,
> > > > > > > > > > > > delete.retention.ms -> 86400000,
> index.interval.bytes
> > ->
> > > > > 4096,
> > > > > > > > > > > > retention.bytes -> 50000000000, min.insync.replicas
> ->
> > 1,
> > > > > > > > > > > > cleanup.policy -> delete,
> > unclean.leader.election.enable ->
> > > > > true,
> > > > > > > > > > > > segment.ms -> 604800000, max.message.bytes ->
> 1000012,
> > > > > > > > > flush.messages
> > > > > > > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio ->
> > 0.5,
> > > > > > > > > retention.ms
> > > > > > > > > > > > -> 86400000, segment.jitter.ms -> 0}.
> > > > > > > > > > > > 2015-01-26 07:02:14,540  INFO
> [kafka-request-handler-5]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,540  INFO
> [kafka-request-handler-5]
> > > > > log.Log -
> > > > > > > > > > > > Truncating log mytopic-0 to offset 0.
> > > > > > > > > > > > 2015-01-26 07:02:14,547  INFO
> [kafka-request-handler-5]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Added fetcher for partitions List([[mytopic,0],
> > initOffset
> > > > 0
> > > > > to
> > > > > > > > > broker
> > > > > > > > > > > > id:7,host:mybroker:54321] )
> > > > > > > > > > > >
> > > > > > > > > > > > Here’s the log in the other broker with a replica of
> > the
> > > > > topic.
> > > > > > > > > > > >
> > > > > > > > > > > > 2015-01-26 07:02:14,312  INFO
> [kafka-request-handler-7]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 7]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [kafka-request-handler-2]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 7]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [kafka-request-handler-2]
> > > > > > > > > > > > log.OffsetIndex - Deleting index
> > > > > > > > > > > > /mypath/mytopic-0/00000000000000000000.index
> > > > > > > > > > > > 2015-01-26 07:02:14,313  INFO
> [kafka-request-handler-2]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Deleted log for partition [mytopic,0] in
> > > > /mypath/mytopic-0.
> > > > > > > > > > > > 2015-01-26 07:02:14,530  INFO
> [kafka-request-handler-0]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 7]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,532  INFO
> [kafka-request-handler-0]
> > > > > log.Log -
> > > > > > > > > > > > Completed load of log mytopic-0 with log end offset 0
> > > > > > > > > > > > 2015-01-26 07:02:14,533  INFO
> [kafka-request-handler-0]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Created log for partition [mytopic,0] in /mypath
> with
> > > > > > > properties
> > > > > > > > > > > > {segment.index.bytes -> 10485760,
> file.delete.delay.ms
> > ->
> > > > > 60000,
> > > > > > > > > > > > segment.bytes -> 1073741824, flush.ms ->
> > > > > 9223372036854775807,
> > > > > > > > > > > > delete.retention.ms -> 86400000,
> index.interval.bytes
> > ->
> > > > > 4096,
> > > > > > > > > > > > retention.bytes -> 50000000000, min.insync.replicas
> ->
> > 1,
> > > > > > > > > > > > cleanup.policy -> delete,
> > unclean.leader.election.enable ->
> > > > > true,
> > > > > > > > > > > > segment.ms -> 604800000, max.message.bytes ->
> 1000012,
> > > > > > > > > flush.messages
> > > > > > > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio ->
> > 0.5,
> > > > > > > > > retention.ms
> > > > > > > > > > > > -> 86400000, segment.jitter.ms -> 0}.
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Jan 26, 2015 at 1:26 AM, Jun Rao <
> > jun@confluent.io
> > > > >
> > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Could you do a get on /admin/deleted_topics and see
> if
> > its
> > > > > > > cversion
> > > > > > > > > > > > changes
> > > > > > > > > > > > > after issuing the delete topic command? This will
> > tell us
> > > > > > > whether
> > > > > > > > > the
> > > > > > > > > > > > > path /admin/deleted_topics//mytopic
> > > > > > > > > > > > > is actually created or not. Anything in the
> > controller
> > > > log
> > > > > that
> > > > > > > > > > > indicates a
> > > > > > > > > > > > > delete topic event is fired?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Jun
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Sun, Jan 25, 2015 at 9:55 PM, Jason Rosenberg <
> > > > > > > > jbr@squareup.com
> > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > yes
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Mon, Jan 26, 2015 at 12:18 AM, Jun Rao <
> > > > > jun@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Do you have delete.topic.enable turned on in
> all
> > > > > brokers?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Jun
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Sun, Jan 25, 2015 at 7:56 PM, Jason
> Rosenberg
> > <
> > > > > > > > > jbr@squareup.com
> > > > > > > > > > > >
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > So far, I have been unable to get delete
> topic
> > to
> > > > > work,
> > > > > > > > with
> > > > > > > > > > > release
> > > > > > > > > > > > > > > > candidate 2 for 0.8.2.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > It worked ok when I ran it in the debugger
> > locally,
> > > > > on a
> > > > > > > > > single
> > > > > > > > > > > node
> > > > > > > > > > > > > > > > instance. But when I run it in our staging
> > > > > environment,
> > > > > > > it
> > > > > > > > > is not
> > > > > > > > > > > > > > > > successfully even marking the topic for
> > delete, for
> > > > > some
> > > > > > > > > reason.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I am setting delete.topic.enable to true (and
> > see
> > > > the
> > > > > > > > > > > confirmation of
> > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > in the startup logs, e.g.):
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > INFO [main] utils.VerifiableProperties -
> > Property
> > > > > > > > > > > delete.topic.enable
> > > > > > > > > > > > > > > > is overridden to true
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I run this command:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand
> > > > --zookeeper
> > > > > > > > > > > > > > > > myzkconnect:12345/mynamespace --delete
> --topic
> > > > > mytopic
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection).
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread).
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > Topic mytopic is marked for deletion.
> > > > > > > > > > > > > > > > Note: This will have no impact if
> > > > > delete.topic.enable is
> > > > > > > > not
> > > > > > > > > set
> > > > > > > > > > > to
> > > > > > > > > > > > > > true.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I then do a —list which should at least show
> > the
> > > > > topic
> > > > > > > > > marked for
> > > > > > > > > > > > > > > deletion:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand
> > > > --zookeeper
> > > > > > > > > > > > > > > > myzkconnect:12345/mynamespace --list --topic
> > > > mytopic
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection).
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread).
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > mytopic
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Note, it doesn’t list it as ‘marked for
> > deletion’.
> > > > > > > > > Furthermore,
> > > > > > > > > > > > > waiting
> > > > > > > > > > > > > > > > multiple hours still doesn’t result in the
> > topic
> > > > > being
> > > > > > > > > deleted.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The topic has 1 partition, 2 replicas, and no
> > data
> > > > > > > stored.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > In zookeeper, the /admin/deleted_topics/ path
> > is
> > > > > empty.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The zookeeper code looks pretty
> > straightforward,
> > > > but
> > > > > for
> > > > > > > > some
> > > > > > > > > > > reason
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > writing the deleted_topics path.  We are
> > running
> > > > > > > zookeeper
> > > > > > > > > 3.4.6.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Jason
> > > > > > > > > > > > > > > > ​
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > ​
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > -- Guozhang
> > > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message