kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ismael Juma <ism...@juma.me.uk>
Subject Re: Brokers are crash due to __consumer_offsets folder are deleted
Date Sat, 02 Jul 2016 12:08:40 GMT
Hi Peter,

It's a good question why `log.dir` defaults to `/tmp`. I assume it's to
make it easier for people to get started with Kafka, but unsafe defaults
should be avoided as much as possible in my opinion.

Ismael

On Sat, Jul 2, 2016 at 5:15 AM, Peter Davis <davispw@gmail.com> wrote:

>
> Dear Community: why does log.dir default under /tmp?  It is unsafe as a
> default.
>
> -Peter
>
>
> > On Jun 30, 2016, at 11:19 PM, 黄杰斌 <jben.huang@gmail.com> wrote:
> >
> > Hi All,
> >
> > Do you encounter below issue when using kafka_2.11-0.10.0.0?
> > All brokers are crash due to __consumer_offsets folder are deleted.
> > sample log:
> > [2016-06-30 12:46:32,579] FATAL [Replica Manager on Broker 2]: Halting
> due
> > to unrecoverable I/O error while handling produce request:
> > (kafka.server.ReplicaManager)
> > kafka.common.KafkaStorageException: I/O exception in append to log
> > '__consumer_offsets-32'
> >        at kafka.log.Log.append(Log.scala:329)
> >        at kafka.cluster.Partition$$anonfun$11.apply(Partition.scala:443)
> >        at kafka.cluster.Partition$$anonfun$11.apply(Partition.scala:429)
> >        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
> >        at kafka.utils.CoreUtils$.inReadLock(CoreUtils.scala:237)
> >        at
> > kafka.cluster.Partition.appendMessagesToLeader(Partition.scala:429)
> >        at
> >
> kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(ReplicaManager.scala:406)
> >        at
> >
> kafka.server.ReplicaManager$$anonfun$appendToLocalLog$2.apply(ReplicaManager.scala:392)
> >        at
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> >        at
> >
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
> >        at scala.collection.immutable.Map$Map1.foreach(Map.scala:116)
> >        at
> > scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
> >        at scala.collection.AbstractTraversable.map(Traversable.scala:104)
> >        at
> > kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.scala:392)
> >        at
> > kafka.server.ReplicaManager.appendMessages(ReplicaManager.scala:328)
> >        at
> >
> kafka.coordinator.GroupMetadataManager.store(GroupMetadataManager.scala:232)
> >        at
> >
> kafka.coordinator.GroupCoordinator$$anonfun$handleCommitOffsets$9.apply(GroupCoordinator.scala:424)
> >        at
> >
> kafka.coordinator.GroupCoordinator$$anonfun$handleCommitOffsets$9.apply(GroupCoordinator.scala:424)
> >        at scala.Option.foreach(Option.scala:257)
> >        at
> >
> kafka.coordinator.GroupCoordinator.handleCommitOffsets(GroupCoordinator.scala:424)
> >        at
> > kafka.server.KafkaApis.handleOffsetCommitRequest(KafkaApis.scala:310)
> >        at kafka.server.KafkaApis.handle(KafkaApis.scala:84)
> >        at
> > kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
> >        at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.io.FileNotFoundException:
> > /tmp/kafka2-logs/__consumer_offsets-32/00000000000000000000.index (No
> such
> > file or directory)
> >        at java.io.RandomAccessFile.open0(Native Method)
> >        at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
> >        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
> >        at
> > kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:286)
> >        at
> > kafka.log.OffsetIndex$$anonfun$resize$1.apply(OffsetIndex.scala:285)
> >        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
> >        at kafka.log.OffsetIndex.resize(OffsetIndex.scala:285)
> >        at
> >
> kafka.log.OffsetIndex$$anonfun$trimToValidSize$1.apply$mcV$sp(OffsetIndex.scala:274)
> >        at
> >
> kafka.log.OffsetIndex$$anonfun$trimToValidSize$1.apply(OffsetIndex.scala:274)
> >        at
> >
> kafka.log.OffsetIndex$$anonfun$trimToValidSize$1.apply(OffsetIndex.scala:274)
> >        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:231)
> >        at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:273)
> >        at kafka.log.Log.roll(Log.scala:655)
> >        at kafka.log.Log.maybeRoll(Log.scala:630)
> >        at kafka.log.Log.append(Log.scala:383)
> >        ... 23 more
> >
> > No one remove those folders, and topic __consumer_offsets is handled by
> > broker, no one can remove this topic.
> > Do you know why this happened? And how to avoid it?
> >
> > Best Regards,
> > Ben
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message