lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Lukyanchikov <alexanderlukyanchi...@gmail.com>
Subject CorruptIndexException after failed segment merge caused by No space left on device
Date Wed, 24 Mar 2021 05:41:20 GMT
Hello everyone,

Recently we had a failed segment merge caused by "No space left on device".
After restart, Lucene failed with the CorruptIndexException.
The expectation was that Lucene automatically recovers in such
case, because there was no succesul commit. Is it a correct assumption, or
I am missing something?
It would be great to know any recommendations to avoid such situations
in future and be able to recover automatically after restart.

Lucene version is 8.5.0

Failed merge stacktrace:

2021-02-02T08:51:51.679+0000

org.apache.lucene.index.MergePolicy$MergeException:
java.io.IOException: No space left on device

	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)

	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)

Caused by: java.io.IOException: No space left on device

	at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)

	at java.base/sun.nio.ch.FileDispatcherImpl.write(FileDispatcherImpl.java:62)

	at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)

	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:79)

	at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)

	at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)

	at java.base/java.nio.channels.Channels.writeFully(Channels.java:97)

	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)

	at org.apache.lucene.store.FSDirectory$FSIndexOutput$1.write(FSDirectory.java:416)

	at java.base/java.util.zip.CheckedOutputStream.write(CheckedOutputStream.java:74)

	at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)

	at java.base/java.io.BufferedOutputStream.write(BufferedOutputStream.java:127)

	at org.apache.lucene.store.OutputStreamIndexOutput.writeBytes(OutputStreamIndexOutput.java:53)

	at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)

	at org.apache.lucene.util.compress.LZ4.encodeLiterals(LZ4.java:159)

	at org.apache.lucene.util.compress.LZ4.encodeSequence(LZ4.java:172)

	at org.apache.lucene.util.compress.LZ4.compress(LZ4.java:441)

	at org.apache.lucene.codecs.compressing.CompressionMode$LZ4FastCompressor.compress(CompressionMode.java:165)

	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.flush(CompressingStoredFieldsWriter.java:229)

	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finishDocument(CompressingStoredFieldsWriter.java:159)

	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.merge(CompressingStoredFieldsWriter.java:636)

	at org.apache.lucene.index.SegmentMerger.mergeFields(SegmentMerger.java:229)

	at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:106)

	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4463)

	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4057)

	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)

	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)


Followed by failed startup:

2021-02-02T08:52:07.926+0000

org.apache.lucene.index.CorruptIndexException: Unexpected file read
error while reading index.
(resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/data/5f91aa0b07ce4d5e7beffaa2/segments_578fu")))

	at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:291)

	at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:846)

Caused by: java.nio.file.NoSuchFileException:
/data/5f91aa0b07ce4d5e7beffaa2/_6lfem.si

	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)

	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)

	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)

	at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:182)

	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)

	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)

	at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:81)

	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)

	at org.apache.lucene.codecs.lucene70.Lucene70SegmentInfoFormat.read(Lucene70SegmentInfoFormat.java:91)

	at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:353)

	at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:289)

	... 33 common frames omitted


Thank you!

--
Regards,
Alexander L

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message