lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Replication lag after cache optimizations
Date Sun, 02 Sep 2012 01:35:33 GMT
You polling interval is much too short. 1 second
is probably getting you into resource contention
issues.

A more reasonable interval is on the order of several
minutes. If you really need near real time
searching, consider 4.0 which supports NRT

Best
Erick

On Fri, Aug 31, 2012 at 10:02 AM, Damien Dudognon
<damien.dudognon@ebuzzing.com> wrote:
> Hi,
>
> We get some troubles with the solr replication after cache optimizations. We use a lot
the facet features.
>
> We have increased the cache size and its initial size. We have also change the queryResultCache
from LRU to FastLRU and the fieldValueCache was activated (see below the detailed configuration).
This optimization step allows to  divide the average time per request by 50 (from 260ms to
5ms).
>
> However, with these modifications we noticed an important replication lag. This issue
is not troublesome for smaller indexes (about 300.000 elements - 3Gb), but it becomes critical
if the index size is significant (30 million elements - 70Gb). In fact, the slaves can't make
up for lag and they become out-of-date (and consequently unusable).
>
> Has anyone ever been faced this kind of problems?
>
> Our environment :
> - Solr 3.4.0
> - Java 1.6.0_26
> - Debian 6.0.3
>
> Best regards,
> Damien
>
> ------------------------------------------------------------------
> My previous cache settings (fieldValueCache was disabled):
> ------------------------------------------------------------------
> <filterCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"
/>
> <queryResultCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"
/>
> <documentCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"
/>
> <!--
> <fieldValueCache class="solr.FastLRUCache" size="512" autowarmCount="128" showItems="32"
/>
> -->
>
> ------------------------------------------------------------------
> The settings now used:
> ------------------------------------------------------------------
> <filterCache class="solr.FastLRUCache" size="16384" initialSize="4096" autowarmCount="0"
/>
> <queryResultCache class="solr.FastLRUCache" size="16384" initialSize="4096" autowarmCount="0"
/>
> <documentCache class="solr.FastLRUCache" size="16384" initialSize="4096" autowarmCount="0"
/>
> <fieldValueCache class="solr.FastLRUCache" size="16384" autowarmCount="1024" showItems="32"
/>
>
> ------------------------------------------------------------------
> The replication config:
> ------------------------------------------------------------------
>   <requestHandler name="/replication" class="solr.ReplicationHandler" >
>     <lst name="master">
>       <str name="enable">${solr.enable.master:false}</str>
>       <str name="replicateAfter">commit</str>
>       <str name="replicateAfter">startup</str>
>       <str name="confFiles">schema.xml,stopwords.txt</str>
>     </lst>
>     <lst name="slave">
>       <str name="enable">${solr.enable.slave:false}</str>
>       <str name="masterUrl">http://solrmaster:${jetty.port:8083}/solr/en/replication</str>
>       <str name="pollInterval">00:00:01</str>
>     </lst>
>   </requestHandler>

Mime
View raw message