lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <>
Subject Re: How much disk space does optimize really take
Date Wed, 07 Oct 2009 16:59:40 GMT
On Wed, Oct 7, 2009 at 12:51 PM, Phillip Farber <> wrote:
> In a separate thread, I've detailed how an optimize is taking > 2x disk
> space. We don't use solr distribution/snapshooter.  We are using the default
> deletion policy = 1. We can't optimize a 192G index in 400GB of space.
> This thread in lucene/java-user
> suggests that an optimize should not take > 2x unless perhaps an IndexReader
> is holding on to segments. This could be our problem since when optimization
> runs out of space, if we stop tomcat, a number of files go away and space is
> recovered.
> But we are not searching the index so how could a Searcher/IndexReader have
> any segments open?
> I notice in the logs that as part of routine commits or as part of optimize
> a Searcher is registered and autowarmed from a previous searcher (of course
> there's nothing in the caches -- this is just a build machine).
> INFO: registering core:
> Oct 6, 2009 2:16:20 PM org.apache.solr.core.SolrCore registerSearcher
> INFO: [] Registered new searcher Searcher@2e097617 main
> Does this means that there's always a lucene IndexReader holding segment
> files open so they can't be deleted during an optimize so we run out of disk
> space > 2x?

A feature could probably now be developed now that avoids opening a
reader until it's requested.
That wasn't really possible in the past - due to many issues such as
Lucene autocommit.


View raw message