lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ishan Chattopadhyaya <ichattopadhy...@gmail.com>
Subject Re: Race condition with RTGs during soft commit
Date Wed, 17 Feb 2016 19:55:24 GMT
JIRA is back up, and I filed SOLR-8687.
Thanks.

On Thu, Feb 18, 2016 at 1:19 AM, Ishan Chattopadhyaya <
ichattopadhyaya@gmail.com> wrote:

> I am facing a problem with stress testing SOLR-5944, even though I think
> this problem persists in Solr even without my changes.
>
> The symptom is that during a stress test (similar to TestStressReorder),
> RTG gets a document which is older version than that of the last
> acknowledged write.
>
> Possible reason:
>
> ```(DUH2's commit())
> ...
> 1:      if (cmd.softCommit) {
> 2:        // ulog.preSoftCommit();
> 3:        synchronized (solrCoreState.getUpdateLock()) {
> 4:          if (ulog != null) ulog.preSoftCommit(cmd);
> 5:          core.getSearcher(true, false, waitSearcher, true);
> 6:          if (ulog != null) ulog.postSoftCommit(cmd);
> 7:        }
> 8:        callPostSoftCommitCallbacks();
> 9:      }
> ...
> ```
>
>
> * Before line 1, there was an update (say id=2) which was in ulog's map.
> Maps are, say, `map={2=LogPtr(1234)}, prevMap={...}, prevMap2={...}`
> * Due to line 4 (ulog.preSoftCommit()), the maps were rotated. Now, the
> id=2 is in prevMap: `map={}, prevMap={2=LogPtr(1234)}, prevMap2={...}`.
> Till now RTG for id=2 will work.
> * Due to line 5, a new searcher is due to be opened. But this is
> asynchronous, and lets assume this doesn't complete before few more lines
> are executed.
> * Due to line 6 (ulog.postSoftCommit()), the previous maps are cleared
> out. Now the maps are: `map={}, prevMap=null, prevMap2=null`
> * If there's an RTG for id=2, it will not work from the ulog's maps, so it
> will fall through to be searched using the last searcher. But, the searcher
> due to be opened in line 5 hasn't yet been opened. In this case, the
> returned document will be whatever version of id=2 that was present in the
> previous searcher.
>
> I cannot seem to access JIRA at the moment, so cannot file a bug right
> now. Can someone please confirm if this is a valid problem? If so, any
> suggestions for a fix, please? (I tried opening a
> ulog.openRealtimeSearcher() in the above synchronized block, but the
> problem still persists).
>

Mime
View raw message