lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-5783) Can we stop opening a new searcher when the index hasn't changed?
Date Tue, 04 Mar 2014 16:56:23 GMT

    [ https://issues.apache.org/jira/browse/SOLR-5783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919610#comment-13919610
] 

Hoss Man commented on SOLR-5783:
--------------------------------

bq. How do you trigger a new searcher on when the Solr instance receiving the commit isn't
responsible for updating the index, such as when it's sharing an index on a shared disk with
another Solr core that is doing the updates?

[~dsmiley]: That type of situation should not be affected at all -- what changed here is that
if {{DirectoryReader.openIfChanged(...)}} returns {{null}} to indicate that the underlying
index hasn't changed, and the same reader should be re-used, then we also re-use the same
searcher tha already wraps that reader.  (This is why the replication tests still work with
this change)

Of course: If you see a problem with some edge case in your setup, please, _PLEASE_ let me
know ASAP so i can try to dig into it.

> Can we stop opening a new searcher when the index hasn't changed?
> -----------------------------------------------------------------
>
>                 Key: SOLR-5783
>                 URL: https://issues.apache.org/jira/browse/SOLR-5783
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: Hoss Man
>             Fix For: 4.8, 5.0
>
>         Attachments: SOLR-5783.patch, SOLR-5783.patch, SOLR-5783.patch, SOLR-5783.patch
>
>
> I've been thinking recently about how/when we re-open searchers -- and what the overhead
of that is in terms of caches and what not -- even if the underlying index hasn't changed.
 
> The particular real world case that got me thinking about this recently is when a deleteByQuery
gets forwarded to all shards in a collection, and then the subsequent (soft)Commit (either
auto or explicit) opens a new searcher -- even if that shard was completley uneffected by
the delete.
> It got me wondering: why don't re-use the same searcher when the index is unchanged?
> From what I can tell, we're basically 99% of the way there (in {{<nrtMode/>}})...
> * IndexWriter.commit is already smart enough to short circut if there's nothing to commit
> * SolrCore.openNewSearcher already uses DirectoryReader.openIfChanged to see if the reader
can be re-used.
> * for "realtime" purposes, SolrCore.openNewSearcher will return the existing searcher
if it exists and the DirectoryReader hasn't changed
> ...The only reason I could think of for not _always_ re-using the same searcher when
the underlying DirectoryReader is identical (ie: that last bullet above) is in the situation
where the "live" schema has changed -- but that seems pretty trivial to account for.
> Is there any other reason why this wouldn't be a good idea for improving performance?



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message