lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clemens Wyss DEV <clemens...@mysign.ch>
Subject AW: [lucene 4.6] NPE when calling IndexReader#openIfChanged
Date Mon, 19 May 2014 10:14:00 GMT
Mike,
first of all thanks for all your input, I really appreciate (as much as I like reading your
blog).

> Hmm, but you swap these files over while an IndexReader is still open on the index?
no IndexReader is open while swapping. At least not by design. We have at most one (current)reader
per Index, which is closed just before swapping. I will investigate on this more deeply, though,
me not being impeccable ;)

> Instead, use IndexWriter.deleteAll, then index all your new docs, 
>then commit, then reopen the reader.
we were facing memory issues and hence decided to do "intermediate commits" (i.e. flush the
writer). And in order to still provide "atomicity" (with respect to a full re-index) we decided
to go the before mentionend approach.

> Separately, it's better to use a near-real-time IndexReader so 
>you don't have to commit in order to see changes.
What if I were to search right after deleteAll?


-----Urspr√ľngliche Nachricht-----
Von: Michael McCandless [mailto:lucene@mikemccandless.com] 
Gesendet: Montag, 19. Mai 2014 11:05
An: Lucene Users
Betreff: Re: [lucene 4.6] NPE when calling IndexReader#openIfChanged

On Mon, May 19, 2014 at 4:59 AM, Clemens Wyss DEV <clemensdev@mysign.ch> wrote:
>> Are you using doc-values updates?
> Not to my knowledge, i.e. not explicitly

Hmm ok.

>> Are you ever removing files directly from the index directory yourself between reopens?
> Yes. Reindexing an index completely(*) is done in a separate temporary index/folder.
After that we (guarded by a mutex) swap the files, i.e. delete the old ones and then move
over the new index files.
>
> (*)we do this for example after a massimport of index-relevant data (instead of updating
each and every element on ist own).

Hmm, but you swap these files over while an IndexReader is still open on the index?  And then
you call openIfChanged?

If so, that's what's leading to this exception: you shouldn't ever need to overwrite files
at index level.

Instead, use IndexWriter.deleteAll, then index all your new docs, then commit, then reopen
the reader.  Alternatively, open IndexWriter with OpenMode.CREATE on the index, index all
your docs, commit, and reopen.

Separately, it's better to use a near-real-time IndexReader so you don't have to commit in
order to see changes.

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Mime
View raw message