lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (LUCENE-6745) RAMInputStream.clone is not thread safe
Date Fri, 21 Aug 2015 14:11:45 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless reopened LUCENE-6745:
----------------------------------------

Reopening this, I think we should revert it:

I dug into this test failure: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-MacOSX/2604/

{noformat}
[junit4] Suite: org.apache.lucene.store.TestMultiMMap
   [junit4]   2> aug 21, 2015 9:50:09 AM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Thread-428,5,TGRP-TestMultiMMap]
   [junit4]   2> java.lang.AssertionError: java.io.EOFException: seek past EOF: MMapIndexInput(path="/Users/jenkins/workspace/Lucene-Solr-5.x-MacOSX/lucene/build/core/test/J1/temp/lucene.store.TestMultiMMap_F4E5EB2F2F531D5D-001/tempDir-005/randombytes")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([F4E5EB2F2F531D5D]:0)
   [junit4]   2> 	at org.apache.lucene.store.ByteBufferIndexInput.clone(ByteBufferIndexInput.java:259)
   [junit4]   2> 	at org.apache.lucene.store.ByteBufferIndexInput$MultiBufferImpl.clone(ByteBufferIndexInput.java:487)
   [junit4]   2> 	at org.apache.lucene.store.BaseDirectoryTestCase$1.run(BaseDirectoryTestCase.java:1200)
   [junit4]   2> Caused by: java.io.EOFException: seek past EOF: MMapIndexInput(path="/Users/jenkins/workspace/Lucene-Solr-5.x-MacOSX/lucene/build/core/test/J1/temp/lucene.store.TestMultiMMap_F4E5EB2F2F531D5D-001/tempDir-005/randombytes")
   [junit4]   2> 	at org.apache.lucene.store.ByteBufferIndexInput.seek(ByteBufferIndexInput.java:174)
   [junit4]   2> 	at org.apache.lucene.store.ByteBufferIndexInput$MultiBufferImpl.seek(ByteBufferIndexInput.java:504)
   [junit4]   2> 	at org.apache.lucene.store.ByteBufferIndexInput.clone(ByteBufferIndexInput.java:257)
   [junit4]   2> 	... 2 more
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestMultiMMap -Dtests.method=testCloneThreadSafety
-Dtests.seed=F4E5EB2F2F531D5D -Dtests.slow=true -Dtests.locale=da -Dtests.timezone=Europe/Zurich
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
{noformat}

The failure is easy to reproduce, and indeed MMap's IndexInput.clone is not thread safe.

But thinking about it more ... I think this is too much to expect from Directory impls.

Instead, I think the bug is in BKDDVFormat, because it's doing "real stuff" with the original
IndexInput it opened, instead of always using clones to do so ...

> RAMInputStream.clone is not thread safe
> ---------------------------------------
>
>                 Key: LUCENE-6745
>                 URL: https://issues.apache.org/jira/browse/LUCENE-6745
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: Trunk, 5.4
>
>         Attachments: LUCENE-6745.patch
>
>
> This took some time to track down ... it's the root cause of the RangeTree failures that
[~steve_rowe] found at https://issues.apache.org/jira/browse/LUCENE-6697?focusedCommentId=14696999&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14696999
> The problem happens when one thread is using the original IndexInput (RAMInputStream)
from a RAMDirectory, but other threads are also cloning  that IndexInput at the same time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message