lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shai Erera (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5574) NRT Reader close can wipe index it doesn't own
Date Fri, 04 Apr 2014 11:14:21 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13959875#comment-13959875
] 

Shai Erera commented on LUCENE-5574:
------------------------------------

I think this is a good change -- it makes IndexReader more read-only in nature. I.e. previously
it could modify the index, then we made it read-only in that aspect. Now we make it sort of
read-only such that it cannot modify the Directory. So I wonder, if we viewed IndexReader
as a "user" with read-only permissions on the index Directory, we wouldn't even attempt deleting
unused files by it, right? So maybe we should just do that -- never call IFD from an nrt-reader.close()
chain? Then, if you have a write open, it will eventually delete those files, otherwise the
files are there until you open a new writer?

As for MDW, what if on close() it attempted to open a writer w/ OpenMode.OPEN, then close
it, before it verifies there are no leftover files? Then the unused files will get deleted,
and we'd still have a check in MDW that we didn't leave any writer/reader open, or leaking
files?

> NRT Reader close can wipe index it doesn't own
> ----------------------------------------------
>
>                 Key: LUCENE-5574
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5574
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/index
>    Affects Versions: 4.8, 5.0, 4.7.1
>            Reporter: Simon Willnauer
>            Priority: Critical
>             Fix For: 4.8, 5.0
>
>         Attachments: LUCENE-5574.patch, LUCENE-5574.patch, LUCENE-5574.patch
>
>
> Today NRT Readers try to clean up unused files via their IW reference when they are closed.
Yet, if the index writer is already closed another index could have been created on the same
directory which can create the same files as the IW before. For the NRT Reader those files
are not referenced and it will simply wipe them away. If you use this in a replication scenario
where directories are reused this can simply wipe your index away or in combination with the
FSync issue LUCENE-5570 create 0-byte files. I have a test that reproduces this issue



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message