lucene-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "" <>
Subject InderxWriter.optimize() fail
Date Tue, 10 Feb 2009 16:29:55 GMT

We scan web and index pages in lucene. Our index size is in the range of
500K to 1 million documens.  As we index pages, we also call
IndexWriter.optimize after certain time intervals [I believe Lucene also
does optimization in the background ?]. So far it has worked great. But for
just this one scan we noticed that the our index size grew to 90 GB for
about 900K documents [typical index size should be around 17-18GB]. We are
not sure what caused the index to grow this large. Outside of our system,
when we did a forced IndexWriter.optimize() on this 90 GB lucene index, it
indeed shrinked to 17 GB. My question is what may have caused the size to
grow to 90GB? Did the size grow because optimization failed ? Does
optimization fail if there is any foreign file in the lucene index directory
[though we tried optimizing with foreign files in lucene directory, and
lucene still did optimize the index.]

any suggestion, input will be quite valuable.
View this message in context:
Sent from the Lucene - General mailing list archive at

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message