lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phillip Farber <pfar...@umich.edu>
Subject How much disk space does optimize really take
Date Wed, 07 Oct 2009 16:51:47 GMT

In a separate thread, I've detailed how an optimize is taking > 2x disk 
space. We don't use solr distribution/snapshooter.  We are using the 
default deletion policy = 1. We can't optimize a 192G index in 400GB of 
space.

This thread in lucene/java-user

http://www.gossamer-threads.com/lists/lucene/java-user/43475

suggests that an optimize should not take > 2x unless perhaps an 
IndexReader is holding on to segments. This could be our problem since 
when optimization runs out of space, if we stop tomcat, a number of 
files go away and space is recovered.

But we are not searching the index so how could a Searcher/IndexReader 
have any segments open?

I notice in the logs that as part of routine commits or as part of 
optimize a Searcher is registered and autowarmed from a previous 
searcher (of course there's nothing in the caches -- this is just a 
build machine).

INFO: registering core:
Oct 6, 2009 2:16:20 PM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher Searcher@2e097617 main

Does this means that there's always a lucene IndexReader holding segment 
files open so they can't be deleted during an optimize so we run out of 
disk space > 2x?

Phil
hathitrust.org




Mime
View raw message