lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <>
Subject Re: Question about aborting background merges
Date Mon, 03 Dec 2007 09:42:04 GMT

"Michael Busch" <> wrote:

> I have a question about IndexWriter.close(false) and background
> merges. I was going to take a look at the code, but I'm sure that Mike
> knows the answer :-). Let's assume that a long background merge is
> going on and close(false) is called. Then the merges are marked as
> aborted and IndexWriter.close() returns after flushing
> DocumentsWriter's buffers. The background merge threads keep going.
> Now a new IndexWriter is opened and optimize() is called. Can it
> happen that optimize() tries to create a segment with the same name
> the background threads are still working on? Then the new IndexWriter
> would probably hit an IOException? Or would the new IndexWriter use
> different file names for the merged segments?

We should be fine here.  When a merge kicks off, it gets a segment name
by calling newSegmentName().  That method gives the merge the next
segment name, and marks commitPending, for exactly this reason (actually
there's a comment in that method explaining this).

When you then close(false), the segments_N that's flushed records the
fact that this name is "in use" and will not re-assign that name when
you next open a writer on the index.

This was in fact a bug at one point (failing to mark commitPending on
giving out a new segmentName), which one of the unit tests exposed.

Hmm ... looking at the code, I think we should also check whether a
merge was aborted in mergeInit.  It looks like there is a window from
when a merge is handed out until when it is inited such that if the
writer is closed in that window it could result in incorrect re-use of a
segment name.  Michael are you seeing such a case?


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message