lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clemens Wyss DEV <clemens...@mysign.ch>
Subject AW: [lucene 4.6] NPE when calling IndexReader#openIfChanged
Date Wed, 21 May 2014 07:17:12 GMT
> Can you just decrease IW's ramBufferSizeMB to relieve the memory pressure?
+1
Is there something alike for IndexReaders?

-----Urspr√ľngliche Nachricht-----
Von: Michael McCandless [mailto:lucene@mikemccandless.com] 
Gesendet: Montag, 19. Mai 2014 12:19
An: Lucene Users
Betreff: Re: [lucene 4.6] NPE when calling IndexReader#openIfChanged

On Mon, May 19, 2014 at 6:14 AM, Clemens Wyss DEV <clemensdev@mysign.ch> wrote:
> Mike,
> first of all thanks for all your input, I really appreciate (as much as I like reading
your blog).

You're welcome!

>> Hmm, but you swap these files over while an IndexReader is still open on the index?
> no IndexReader is open while swapping. At least not by design. We have 
> at most one (current)reader per Index, which is closed just before 
> swapping. I will investigate on this more deeply, though, me not being 
> impeccable ;)

Hmm, double check that.  The only way I can explain what you're seeing is if the reader was
open on the index, then the index was blown away / replaced, then openIfChanged called ...

>> Instead, use IndexWriter.deleteAll, then index all your new docs, 
>>then commit, then reopen the reader.
> we were facing memory issues and hence decided to do "intermediate commits" (i.e. flush
the writer). And in order to still provide "atomicity" (with respect to a full re-index) we
decided to go the before mentionend approach.

Can you just decrease IW's ramBufferSizeMB to relieve the memory pressure?

>> Separately, it's better to use a near-real-time IndexReader so you 
>>don't have to commit in order to see changes.
> What if I were to search right after deleteAll?

That's fine, it's fully transactional: the deleteAll will not be visible until you open a
new reader.  If you change your mind, you can call rollback after deleteAll and the docs are
back, etc.

Even opening a new IW with OpenMode.CREATE is transactional: that change will not become visible
to a reader until it's reopened.

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Mime
View raw message