lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: "on deck" searcher vs warming searcher
Date Tue, 13 Dec 2016 23:07:05 GMT


(disclaimer: i'm writing this all from memory, maybe there was some code 
change at some point that i'm not aware of and i'm completley wrong)


: I've always understood the "on deck" searcher(s) being the same as the
: warming searcher(s). So you have the "active" searcher and them the warming
: or on deck searchers.

IIRC there is a subtle but significant distinction between "ondeck" 
searchers and warming searchers -- namely: there is only ever one (ondeck) 
searcher being warmed at a time, but there can be multiple ondeck 
searchers *WAITING* to be warmed.

maxWarmingSearchers is in hindsight not the greatest name for that cnfig 
option -- because there's never more then one searcher being warmed at 
a time.  it esentially means "max number of searchers i'm willing to let 
queue up to be warmed"


Or to elaborate and put it all in context:

* There is only ever (at most) one "current" searcher:
** when a request comes in, it grabs a refrence to the current searcher
** when a "newSearcher" event finishes, the "current" searcher can be 
replaced, but the (previous) searcher stays open as long as there are 
requests in progress that have references to it
** for this reason: there can be multiple "active" searchers in use

* There can be any number of searchers ondeck:
** an ondeck searcher is created when a "newSearcher" event starts
** ondeck searchers can be "warmed up" based on the caches of the 
"current" searcher, during which time they are considered 
a "warming searcher"
** "warming" happens i na single threaded executor -- so if there are 
multiple ondeck searchers, only one of them at a time is ever a "warming" 
searcher
** multiple ondeck searchers can be a sign of a potential performance 
problem (hence the log warning) because it means Solr knows you are 
(currently) actively warming a seacher pointed at some commit point which 
solr already knows is stale and plans to open another searcher immediatley 
after it -- ie: commits are happening more frequently then searchers can 
be warmed.

If you imagine a a scenerio where:
* search requests that take over 10 seconds happen continuously all day
* caches are sized such that warming a newSearcher takes 20 seconds
* you set maxWarmingSearchers=15
* openSearcher=true commits happen once per second

Then if i'm not mistaken you would be in a situation where there would be 
10 "active" searchers (the newest of which is "current"), 15 "ondeck" 
searchers (the oldest of which would be currently be a "warming" searcher


-Hoss
http://www.lucidworks.com/

Mime
View raw message