lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anand Bhagwat <abbhagwa...@gmail.com>
Subject Re: Lucene not merging frequently causing huge index size
Date Tue, 28 Apr 2015 03:51:25 GMT
Thanks a lot. I am running my application on windows.

As you suggested I am now closing the indexreader after every read and it
has now solved the problem. My only question is how costly it would be to
close and open the indexreader every time. Is it too heavy in terms of
performance?

-Anand.

On 27 April 2015 at 18:55, Michael McCandless <lucene@mikemccandless.com>
wrote:

> Opening a single IndexWriter and using it 'forever' is the right way
> to do NRT indexing+searching, and merging should keep segment counts
> in check.
>
> Are you on Windows?  Is it possible you're not closing every
> IndexReader you opened?  This will cause files to be held open and
> then IndexWriter can't delete them.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Mon, Apr 27, 2015 at 3:00 AM, Anand Bhagwat <abbhagwatgm@gmail.com>
> wrote:
> > Let me explain the usage scenario:
> > I have an application with frequent doucment updates and it uses NRT
> > reader. I also have a background thread which calls commit on the
> > IndexWriter on a regular interval. There is one instance of IndexWriter
> > which is created and it is never closed.
> >
> > The problem is that Lucene is creating lot of files and huge index which
> is
> > way much more than the size of documents. This size is not reduced
> despite
> > the lucene merge running in the background. (I am using the
> > default TieredMergePolicy and ConcurrentMergeScheduler).
> >
> > I also tried running a background thread calling IndexWriter.maybeMerge()
> > at regular interval still the size of the index is not changing.
> >
> > I then tried calling IndexWriter.close() at regular interval and found
> that
> > this actually reduces the size of index and performs merging.
> >
> > So my question is what is the suggested way of using IndexWriter? Do I
> need
> > to close it on regular intervals?
> > Also why merge which is running in the background is not reducing the
> index
> > size?
> >
> > Thanks,
> > Anand Bhagwat.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message