lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bernd Fehling <bernd.fehl...@uni-bielefeld.de>
Subject Re: old searchers not closing after optimize or replication
Date Wed, 20 Apr 2011 12:37:53 GMT
Hi Erik,

Am 20.04.2011 13:56, schrieb Erick Erickson:
> Does this persist? In other words, if you just watch it for
> some time, does the disk usage go back to normal?

Only after restarting the whole solr the disk usage goes back to normal.

>
> Because it's typical that your index size will temporarily
> spike after the operations you describe as new searchers
> are warmed up. During that interval, both the old and new
> searchers are open.

Temporarily yes, but still after a couple of hours after optimize
or replication?

>
> Look particularly at your warmup time in the Solr admin page,
> that should give you an indication of how long it takes your
> warmup to happen and give you a clue about when you should
> expect the index sizes to drop again.

We have newSearcher and firstSearcher (both with 2 simple queries) and
<useColdSearcher>false</useColdSearcher>
<maxWarmingSearchers>2</maxWarmingSearchers>
The QTime is less than 500 (0.5 second).

warmupTime=0 for all autowarming Searcher

>
> How often do you optimize on the master and replicate on the
> slave? Because you may be getting into the runaway warmup
> problem where a new searcher is opened before the last one
> is autowarmed and spiraling out of control.

We commit new content about every hour and do an optimze once a day.
So replication is also once a day after optimize finished and
system has settled down.
No commit during optimize and replication.


Any further hints?


>
> Hope that helps
> Erick
>
> On Wed, Apr 20, 2011 at 2:36 AM, Bernd Fehling
> <bernd.fehling@uni-bielefeld.de>  wrote:
>> Hello list,
>>
>> we have the problem that old searchers often are not closing
>> after optimize (on master) or replication (on slaves) and
>> therefore have huge index volumes.
>> Only solution so far is to stop and start solr which cleans
>> up everything successfully, but this can only be a workaround.
>>
>> Is the parameter "waitSearcher=false" an option to solve this?
>>
>> Any hints what to check or to debug?
>>
>> We use Apache Solr 3.1.0 on Linux.
>>
>> Regards
>> Bernd
>>

Mime
View raw message