lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Lucene not merging frequently causing huge index size
Date Mon, 27 Apr 2015 12:59:05 GMT
Opening a single IndexWriter and using it 'forever' is the right way
to do NRT indexing+searching, and merging should keep segment counts
in check.

Are you on Windows?  Is it possible you're not closing every
IndexReader you opened?  This will cause files to be held open and
then IndexWriter can't delete them.

Mike McCandless

http://blog.mikemccandless.com


On Mon, Apr 27, 2015 at 7:15 AM, Anand Bhagwat <abbhagwatgm@gmail.com> wrote:
> Let me explain the usage scenario:
> I have an application with frequent doucment updates and it uses NRT
> reader. I also have a background thread which calls commit on the
> IndexWriter on a regular interval. There is one instance of IndexWriter
> which is created and it is never closed.
>
> The problem is that Lucene is creating lot of files and huge index which is
> way much more than the size of documents. This size is not reduced despite
> the lucene merge running in the background. (I am using the
> default TieredMergePolicy and ConcurrentMergeScheduler).
>
> I also tried running a background thread calling IndexWriter.maybeMerge()
> at regular interval still the size of the index is not changing.
>
> I then tried calling IndexWriter.close() at regular interval and found that
> this actually reduces the size of index and performs merging.
>
> So my question is what is the suggested way of using IndexWriter? Do I need
> to close it on regular intervals?
> Also why merge which is running in the background is not reducing the index
> size?
>
>
> Thanks,
> Anand Bhagwat.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message