lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: index size growing while deleting
Date Tue, 10 Nov 2015 15:45:33 GMT
On Tue, Nov 10, 2015 at 7:47 AM, Rob Audenaerde
<rob.audenaerde@gmail.com> wrote:
> Ah yes, that is the way to go.
>
> It is a bit harder here, because we also use a per-user InMemoryIndex that
> is combined in a multi-reader, so it will be a bit more work, but I think
> it will be doable. Thanks for all the help.
>
> That said, I found it not-so-easy to debug this issue; are there methods
> (on the IndexWriter / text in the infoStream?) that I could have used to
> detect what was going on? That might be helpful for other as well?

You can turn on IW's infoStream, but it produces tons of output that's
hard to decipher here.

It comes down to reference counting: each reader open against a
segment holds a ref count on all files for that segment, each commit
point does the same, and so does the "in memory" segments that IW
owns.  It's only once ref count drops to 0 that a file is attempted to
be deleted.

Is it possible you have some readers that are never closed?  Do you
have a separate process opening readers, are are the readers +
IndexWriter all in a single JVM?

I agree better diagnostics here would be nice ... not sure how.

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message