lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emir Arnautovic <emir.arnauto...@sematext.com>
Subject Re: solr indexing warning
Date Fri, 20 Nov 2015 10:40:06 GMT
Hi,
Since this is master node, and not expected to have queries, you can 
disable caches completely. However, from numbers cache autowarm is not 
an issue here but probably frequency of commits and/or warmup queries. 
How do you do commits? Since master-slave, I don't see reason to have 
them too frequently. If you need NRT you should switch to SolrCloud. Do 
you have warmup queries? You don't need them on master node.

Regards,
Emir

On 20.11.2015 08:33, Midas A wrote:
> thanks Shawn,
>
> As we are this server as a master server  there are no queries running on
> it  . in that case should i remove these configuration from config file .
>
> Total Docs: 40 00000
>
> Stats
> #
>
> Document cache :
> lookups:823
> hits:4
> hitratio:0.00
> inserts:820
> evictions:0
> size:820
> warmupTime:0
> cumulative_lookups:24474
> cumulative_hits:1746
> cumulative_hitratio:0.07
> cumulative_inserts:22728
> cumulative_evictions:13345
>
>
> fieldcache:
> stats:
> entries_count:2
> entry#0:'SegmentCoreReader(​owner=_3bph(​4.2.1):C3918553)'=>'_version_',long,org.apache.lucene.search.FieldCache.NUMERIC_UTILS_LONG_PARSER=>org.apache.lucene.search.FieldCacheImpl$LongsFromArray#1919958905
> entry#1:'SegmentCoreReader(​owner=_3bph(​4.2.1):C3918553)'=>'_version_',class
> org.apache.lucene.search.FieldCacheImpl$DocsWithFieldCache,null=>org.apache.lucene.util.Bits$MatchAllBits#660036513
> insanity_count:0
>
>
> fieldValuecache:
>
> lookups:0
>
> hits:0
>
> hitratio:0.00
>
> inserts:0
>
> evictions:0
>
> size:0
>
> warmupTime:0
>
> cumulative_lookups:0
>
> cumulative_hits:0
>
> cumulative_hitratio:0.00
>
> cumulative_inserts:0
>
> cumulative_evictions:0
>
>
> filtercache:
>
>
> lookups:0
>
> hits:0
>
> hitratio:0.00
>
> inserts:0
>
> evictions:0
>
> size:0
>
> warmupTime:0
>
> cumulative_lookups:0
>
> cumulative_hits:0
>
> cumulative_hitratio:0.00
>
> cumulative_inserts:0
>
> cumulative_evictions:0
>
>
> QueryResultCache:
>
> lookups:3841
>
> hits:0
>
> hitratio:0.00
>
> inserts:4841
>
> evictions:3841
>
> size:1000
>
> warmupTime:213
>
> cumulative_lookups:58438
>
> cumulative_hits:153
>
> cumulative_hitratio:0.00
>
> cumulative_inserts:58285
>
> cumulative_evictions:57285
>
>
>
> Please suggest .
>
>
>
> On Fri, Nov 20, 2015 at 12:15 PM, Shawn Heisey <apache@elyograg.org> wrote:
>
>> On 11/19/2015 11:06 PM, Midas A wrote:
>>> <filterCache class="solr.FastLRUCache" size="5000" initialSize="5000"
>>> autowarmCount="1000"/> <!-- Query Result Cache Caches results of
>> searches -
>>> ordered lists of document ids (DocList) based on a query, a sort, and the
>>> range of documents requested. --> <queryResultCache class="solr.LRUCache"
>>> size="1000" initialSize="1000" autowarmCount="1000"/> <!-- Document Cache
>>> Caches Lucene Document objects (the stored fields for each document).
>> Since
>>> Lucene internal document ids are transient, this cache will not be
>>> autowarmed. --> <documentCache class="solr.LRUCache" size="1000"
>> initialSize
>>> ="1000" autowarmCount="1000"/>
>> Your caches are quite large.  More importantly, your autowarmCount is
>> very large.  How many documents are in each of your cores?  If you check
>> the Plugins/Stats area in the admin UI for your core(s), how many
>> entries are actually in each of those three caches?  Also shown there is
>> the number of milliseconds that it took for each cache to warm.
>>
>> The documentCache cannot be autowarmed, so that config is not doing
>> anything.
>>
>> When a cache is autowarmed, what this does is look up the key for the
>> top N entries in the old cache, which contains the query used to
>> generate that cache entry, and executes each of those queries on the new
>> index to populate the new cache.
>>
>> This means that up to 2000 queries are being executed every time you
>> commit and open a new searcher.  The actual number may be less, if the
>> filterCache and queryResultCache are not actually reaching 1000 entries
>> each.  Autowarming can take a significant amount of time when the
>> autowarmCount is high.  It should be lowered.
>>
>> Thanks,
>> Shawn
>>
>>

-- 
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/


Mime
View raw message