kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Birdsong <david.birds...@gmail.com>
Subject Re: no space left error
Date Tue, 06 Jan 2015 20:10:14 GMT
I'm keen to hear about how to work one's way out of a filled partition
since I've run into this many times after having tuned retention bytes or
retention (time?) incorrectly. The proper path to resolving this isn't
obvious based on my many harried searches through documentation.

I often end up stopping the particular broker, picking an unlucky
topic/partition, deleting, modifying the any topics that consumed too much
space by lowering their retention bytes, and restarting.

On Tue, Jan 6, 2015 at 12:02 PM, Sa Li <salicn@gmail.com> wrote:

> Continue this issue, when I restart the server, like
> bin/kafka-server-start.sh config/server.properties
>
> it will fails to start the server, like
>
> [2015-01-06 20:00:55,441] FATAL Fatal error during KafkaServerStable
> startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> java.lang.InternalError: a fault occurred in a recent unsafe memory access
> operation in compiled Java code
>         at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>         at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>         at
> kafka.log.FileMessageSet$$anon$1.makeNext(FileMessageSet.scala:188)
>         at
> kafka.log.FileMessageSet$$anon$1.makeNext(FileMessageSet.scala:165)
>         at
> kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66)
>         at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58)
>         at kafka.log.LogSegment.recover(LogSegment.scala:165)
>         at kafka.log.Log.recoverLog(Log.scala:179)
>         at kafka.log.Log.loadSegments(Log.scala:155)
>         at kafka.log.Log.<init>(Log.scala:64)
>         at
>
> kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$4.apply(LogManager.scala:118)
>         at
>
> kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$4.apply(LogManager.scala:113)
>         at
>
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
>         at
> scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:105)
>         at
> kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:113)
>         at
> kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:105)
>         at
>
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
>         at
> scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)
>         at kafka.log.LogManager.loadLogs(LogManager.scala:105)
>         at kafka.log.LogManager.<init>(LogManager.scala:57)
>         at kafka.server.KafkaServer.createLogManager(KafkaServer.scala:275)
>         at kafka.server.KafkaServer.startup(KafkaServer.scala:72)
>         at
> kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
>         at kafka.Kafka$.main(Kafka.scala:46)
>         at kafka.Kafka.main(Kafka.scala)
> [2015-01-06 20:00:55,443] INFO [Kafka Server 100], shutting down
> (kafka.server.KafkaServer)
> [2015-01-06 20:00:55,444] INFO Terminate ZkClient event thread.
> (org.I0Itec.zkclient.ZkEventThread)
> [2015-01-06 20:00:55,446] INFO Session: 0x684a5ed9da3a1a0f closed
> (org.apache.zookeeper.ZooKeeper)
> [2015-01-06 20:00:55,446] INFO EventThread shut down
> (org.apache.zookeeper.ClientCnxn)
> [2015-01-06 20:00:55,447] INFO [Kafka Server 100], shut down completed
> (kafka.server.KafkaServer)
> [2015-01-06 20:00:55,447] INFO [Kafka Server 100], shutting down
> (kafka.server.KafkaServer)
>
> Any ideas
>
> On Tue, Jan 6, 2015 at 12:00 PM, Sa Li <salicn@gmail.com> wrote:
>
> > the complete error message:
> >
> > -su: cannot create temp file for here-document: No space left on device
> > OpenJDK 64-Bit Server VM warning: Insufficient space for shared memory
> > file:
> >    /tmp/hsperfdata_root/19721
> > Try using the -Djava.io.tmpdir= option to select an alternate temp
> > location.
> > [2015-01-06 19:50:49,244] FATAL  (kafka.Kafka$)
> > java.io.FileNotFoundException: conf (No such file or directory)
> >         at java.io.FileInputStream.open(Native Method)
> >         at java.io.FileInputStream.<init>(FileInputStream.java:146)
> >         at java.io.FileInputStream.<init>(FileInputStream.java:101)
> >         at kafka.utils.Utils$.loadProps(Utils.scala:144)
> >         at kafka.Kafka$.main(Kafka.scala:34)
> >         at kafka.Kafka.main(Kafka.scala)
> >
> > On Tue, Jan 6, 2015 at 11:58 AM, Sa Li <salicn@gmail.com> wrote:
> >
> >>
> >> Hi, All
> >>
> >> I am doing performance test on our new kafka production server, but
> after
> >> sending some messages (even faked message by using
> bin/kafka-run-class.sh
> >> org.apache.kafka.clients.tools.ProducerPerformance), it comes out the
> error
> >> of connection, and shut down the brokers, after that, I see such errors,
> >>
> >> conf-su: cannot create temp file for here-document: No space left on
> >> device
> >>
> >> How can I fix it, I am concerning that will happen when we start to
> >> publish real messages in kafka, and should I create some cron to
> regularly
> >> clean certain directories?
> >>
> >> thanks
> >>
> >> --
> >>
> >> Alec Li
> >>
> >
> >
> >
> > --
> >
> > Alec Li
> >
>
>
>
> --
>
> Alec Li
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message