lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: "on deck" searcher vs warming searcher
Date Wed, 14 Dec 2016 17:34:49 GMT

: In a situation where searchers A-E are queued in the states
: A: Current
: B: Warming
: C: Ondeck
: D: Ondeck
: E: Being created with newSearcher
: 
: wouldn't it make sense to discard C before it gets promoted to Warming,
: as the immediate action after warming C would be to start warming D?
: 
: Are there some situations where the (potentially extremely short lived)
: C searcher must be visible before D replaces it?

In theory it might make sense to throw out C, but in practice: 

1) since maxWarmingSearchers is typically a small value, E (and 
sometimes D) are rarely created

2) because of how the code is structured, discarding C isn't 
particularly easy ... the calls are happening in parallel threads, ie: 
some Thread#1 is warming B while some thread #2 has just opened C and is 
blocked on the single threaded warming executor while waiting to warm it.  
When Thread #3 comes along and opens D, it also gets blocked on the same 
executor.  

We'd need to revamp that code in some way that the existence of Thread #3 
(and beyond) while Thread #2 is queued up would cause Thread #2 to close C 
(w/o warming it) and instead be blocked waiting for D to warm -- such that 
once D completes warming both Thread #2 and Thread #3 both return D.

All of which is complicated by the fact that the code is actaully 
returning the Searchers immediately, but also returning/setting a Future 
ref that is what's waiting on the warming to finish -- so callers can 
actaully use the searchers concurrently with the warming (ie: 
useColdSearchers) if they wish.


So in a nutshell: yes, but it would be a pretty invasive change, and AFAIK 
rarely impacts people who don't already have bigger problems.




-Hoss
http://www.lucidworks.com/

Mime
View raw message