lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <>
Subject Re: index size before and after commit
Date Thu, 01 Oct 2009 14:15:39 GMT
Nice one ;) Its not technically a case where optimize requires > 2x
though in case the user asking gets confused. Its a case unrelated to
optimize that can grow your index. Then you need < 2x for the optimize,
since you won't copy the deletes.

It also requires that you jump hoops to delete everything. If you delete
everything with *:*, that is smart enough not to just do a delete on
every document - it just creates a new index, allowing the removal of
the old very efficiently.

Def agree on the more disk space.

Walter Underwood wrote:
> Here is how you need 3X. First, index everything and optimize. Then
> delete everything and reindex without any merges.
> You have one full-size index containing only deleted docs, one
> full-size index containing reindexed docs, and need that much space
> for a third index.
> Honestly, disk is cheap, and there is no way to make Lucene work
> reliably with less disk. 1TB is a few hundred dollars. You have a free
> search engine, buy some disk.
> wunder
> On Oct 1, 2009, at 6:25 AM, Grant Ingersoll wrote:
>>> 151GB or as little as from 183GB to 182GB.  Is that size after a
>>> commit close to the size the index would be after an optimize?  For
>>> that matter, are there cases where optimization can take more than
>>> 2x?  I've heard of cases but have not observed them in my system.
>> I seem to recall a case where it can be 3x, but I don't know that it
>> has been observed much.

- Mark

View raw message