lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5783) Can we stop opening a new searcher when the index hasn't changed?
Date Sat, 08 Mar 2014 15:57:42 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924904#comment-13924904
] 

Yonik Seeley commented on SOLR-5783:
------------------------------------

Looking back at the original usecase:
bq. The particular real world case that got me thinking about this recently is when a deleteByQuery
gets forwarded to all shards in a collection, and then the subsequent (soft)Commit (either
auto or explicit) opens a new searcher – even if that shard was completley uneffected by
the delete.
And...
bq. IndexWriter.commit is already smart enough to short circut if there's nothing to commit
If that's the case, why is there anything else to do?  Oh, maybe DUH2 no longer short circuits?
(I'm pretty sure I had code in there at one time to do nothing if nothing had changed)
short circuiting at commit() rather than getSearcher/openSearcher seems a lot easier and less
risky.




> Can we stop opening a new searcher when the index hasn't changed?
> -----------------------------------------------------------------
>
>                 Key: SOLR-5783
>                 URL: https://issues.apache.org/jira/browse/SOLR-5783
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Hoss Man
>             Fix For: 4.8, 5.0
>
>         Attachments: SOLR-5783.patch, SOLR-5783.patch, SOLR-5783.patch, SOLR-5783.patch
>
>
> I've been thinking recently about how/when we re-open searchers -- and what the overhead
of that is in terms of caches and what not -- even if the underlying index hasn't changed.
 
> The particular real world case that got me thinking about this recently is when a deleteByQuery
gets forwarded to all shards in a collection, and then the subsequent (soft)Commit (either
auto or explicit) opens a new searcher -- even if that shard was completley uneffected by
the delete.
> It got me wondering: why don't re-use the same searcher when the index is unchanged?
> From what I can tell, we're basically 99% of the way there (in {{<nrtMode/>}})...
> * IndexWriter.commit is already smart enough to short circut if there's nothing to commit
> * SolrCore.openNewSearcher already uses DirectoryReader.openIfChanged to see if the reader
can be re-used.
> * for "realtime" purposes, SolrCore.openNewSearcher will return the existing searcher
if it exists and the DirectoryReader hasn't changed
> ...The only reason I could think of for not _always_ re-using the same searcher when
the underlying DirectoryReader is identical (ie: that last bullet above) is in the situation
where the "live" schema has changed -- but that seems pretty trivial to account for.
> Is there any other reason why this wouldn't be a good idea for improving performance?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message