kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lawrence Craft <lawrence.cr...@gmail.com>
Subject Re: Issue deleting topic on Windows
Date Thu, 22 Mar 2018 19:27:25 GMT
Hi Ben,

This sounds very similar to KAFKA-1194, an issue we encountered on our
Windows servers at work. Basically, Kafka is unable to rename log files as
they're locked by the OS. The bug is here and is still open:
https://issues.apache.org/jira/browse/KAFKA-1194. I am not aware of any
straightforward workaround.

To get around this, we've had to modify the code and run a custom build.
You can try the changes from here: https://github.com/apache/kafka/pull/4431

Hope this helps!

Lawrence


On Thu, Mar 22, 2018 at 9:34 AM, Young, Ben <Ben.Young@fisglobal.com> wrote:

> Hi,
>
> Any thoughts on the below?
>
> Thanks!
>
> -----Original Message-----
>
> Sorry, I forgot to mention we're using version: kafka_2.12-1.0.0
>
> -----Original Message-----
>
> Hi,
>
> We have an issue in our dev environments when running on Windows. To
> provide a clean environment for testing we tear down the topics and
> recreate them, but deleting the topics is causing the server to crash. We
> then need to delete the logs directory for it to be able to restart, it
> doesn't seem to be able to recover.
>
> The backtrace is below. I wonder if something still has a handle open on
> the folder before the rename is attempted?
>
> Thanks,
> Ben
>
> [2018-03-20 09:31:24,969] INFO [ReplicaFetcherManager on broker 0] Removed
> fetcher for partitions TradeFeed-3 (kafka.server.ReplicaFetcherManager)
> [2018-03-20 09:31:25,012] ERROR Error while deleting TradeFeed-3 in dir
> f:\tmp\kafka-logs. (kafka.server.LogDirFailureChannel)
> java.io.IOException: Failed to rename log directory from
> f:\tmp\kafka-logs\TradeFeed-3 to f:\tmp\kafka-logs\TradeFeed-3.
> 23a377d8b9b042a9853adc5deec53a2f-delete
>         at kafka.log.LogManager.asyncDelete(LogManager.scala:671)
>         at kafka.cluster.Partition.$anonfun$delete$1(Partition.scala:178)
>         at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:217)
>         at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:225)
>         at kafka.cluster.Partition.delete(Partition.scala:173)
>         at kafka.server.ReplicaManager.stopReplica(ReplicaManager.
> scala:341)
>         at kafka.server.ReplicaManager.$anonfun$stopReplicas$2(
> ReplicaManager.scala:373)
>         at scala.collection.Iterator.foreach(Iterator.scala:929)
>         at scala.collection.Iterator.foreach$(Iterator.scala:929)
>         at scala.collection.AbstractIterator.foreach(Iterator.scala:1417)
>         at scala.collection.IterableLike.foreach(IterableLike.scala:71)
>         at scala.collection.IterableLike.foreach$(IterableLike.scala:70)
>         at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>         at kafka.server.ReplicaManager.stopReplicas(ReplicaManager.
> scala:371)
>         at kafka.server.KafkaApis.handleStopReplicaRequest(
> KafkaApis.scala:190)
>         at kafka.server.KafkaApis.handle(KafkaApis.scala:104)
>         at kafka.server.KafkaRequestHandler.run(
> KafkaRequestHandler.scala:65)
>         at java.lang.Thread.run(Unknown Source)
> [2018-03-20 09:31:25,013] INFO [ReplicaManager broker=0] Stopping serving
> replicas in dir f:\tmp\kafka-logs (kafka.server.ReplicaManager)
>
>
> The information contained in this message is proprietary and/or
> confidential. If you are not the intended recipient, please: (i) delete the
> message and all copies; (ii) do not disclose, distribute or use the message
> in any manner; and (iii) notify the sender immediately. In addition, please
> be aware that any message addressed to our domain is subject to archiving
> and review by persons other than the intended recipient. Thank you.
> The information contained in this message is proprietary and/or
> confidential. If you are not the intended recipient, please: (i) delete the
> message and all copies; (ii) do not disclose, distribute or use the message
> in any manner; and (iii) notify the sender immediately. In addition, please
> be aware that any message addressed to our domain is subject to archiving
> and review by persons other than the intended recipient. Thank you.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message