lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5673) MmapDirectory shouldn't pass along OOM wrapped as IOException
Date Fri, 16 May 2014 12:52:15 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13999776#comment-13999776
] 

Uwe Schindler commented on LUCENE-5673:
---------------------------------------

I tried the latest patch with Linux 32 bit and an {{ulimit -v 1_000_000}}:

{noformat}
   [junit4] ERROR    132s | Test4GBStoredFields.test <<<
   [junit4]    > Throwable #1: java.io.IOException: Map failed: MMapIndexInput(path="/media/sf_Host/Projects/lucene/trunk-lusolr1/lucene/build/core/test/J0/lucene.index.Test4GBStoredFields-C16129C282E2746E-001/4GBStoredFields-001/_0.fdt")
[this may be caused by lack of enough unfragmented virtual address space or too restrictive
virtual memory limits enforced by the operating system, preventing us to map a chunk of 268435456
bytes. MMapDirectory should only be used on 64bit platforms, because the address space on
32bit operating systems is too small. More information: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C16129C282E2746E:493516182C1E1996]:0)
   [junit4]    > 	at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:888)
   [junit4]    > 	at org.apache.lucene.store.MMapDirectory.map(MMapDirectory.java:271)
   [junit4]    > 	at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:221)
   [junit4]    > 	at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:196)
   [junit4]    > 	at org.apache.lucene.store.Directory.copy(Directory.java:187)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.copy(MockDirectoryWrapper.java:947)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.copy(TrackingDirectoryWrapper.java:50)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4504)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:485)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:452)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:518)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:629)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3042)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3018)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1671)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1647)
   [junit4]    > 	at org.apache.lucene.index.Test4GBStoredFields.test(Test4GBStoredFields.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
{noformat}

> MmapDirectory shouldn't pass along OOM wrapped as IOException
> -------------------------------------------------------------
>
>                 Key: LUCENE-5673
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5673
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: LUCENE-5673.patch, LUCENE-5673.patch, LUCENE-5673.patch, LUCENE-5673.patch
>
>
> The bug here is in java (not MMapDir), but i think we shoudl do something.
> Users get confused when they configure their JVM to trigger something on OOM, and then
see "OutOfMemoryError: Map Failed": but their trigger doesnt fire.
> Thats because in the jdk, when it maps files it catches OutOfMemoryError, asks for a
garbage collection, sleeps for 100 milliseconds, then tries to map again. if it fails a second
time it wraps the OOM in a generic IOException.
> I think we should add a try/catch to our filechannel.map



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message