lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <>
Subject Re: weird mergeFactor behavior in v1.4.3
Date Sat, 05 May 2007 05:00:25 GMT

: up into 1 segment of 1000 docs. So far so good. However if I add 900 docs to
: the index and then add 100 docs in a second pass (closing the IndexWriter
: between the two runs), I get 10 segments on disk but they are not merged.
: Also, if I add 10 docs with mergeFactor=1, I get 10 segments.
: Is this expected? I would have thought that a) segments are merged as they
: are created regardless of timing b) a mergeFactor=1 would always keep the
: index optimized. Am I running into a bug in version 1.4.3? Any ideas for how
: to get around this? Upgrading to the latest version is not an option in my
: case (I'm at the very end of a release cycle).

I believe you are encountering a known bug with releases prior to 2.1,
noted in CHANGES.txt...

  6. LUCENE-672: New indexing segment merge policy flushes all
     buffered docs to their own segment and delays a merge until
     mergeFactor segments of a certain level have been accumulated.
     This increases indexing performance in the presence of deleted
     docs or partially full segments as well as enabling future

     NOTE: this also fixes an "under-merging" bug whereby it is
     possible to get far too many segments in your index (which will
     drastically slow down search, risks exhausting file descriptor
     limit, etc.).  This can happen when the number of buffered docs
     at close, plus the number of docs in the last non-ram segment is
     greater than mergeFactor. (Ning Li, Yonik Seeley)


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message