lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-7332) Seed version buckets with max version from index
Date Wed, 01 Apr 2015 20:46:53 GMT

    [ https://issues.apache.org/jira/browse/SOLR-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14391417#comment-14391417
] 

Yonik Seeley commented on SOLR-7332:
------------------------------------

Wow, nice results!

We still have some correctness issues though (but fixing them should not impact performance).
VersionBucket.highest must be greater or equal to the highest version encountered for that
bucket.
Having a bucket value too high will only cause a few unnecessary lookups, but having a value
too low will cause correctness issues.

Primary issue: commits happen concurrently with other updates, so attempting to initialize
bucket versions from an "active" index will mean that we will sometimes set a version too
low. "As coded, this doesn't happen until the first soft- or hard- commit is triggered" sounds
like the index will indeed be active.

Secondary issue: there is a race in setting the bucket version.  VersionBucket.updateHighest
will be called with the monitor held, but between the test and the set, VersionInfo.seedBucketVersionHighestFromIndex
can change "highest".  Or vice-versa.

> Seed version buckets with max version from index
> ------------------------------------------------
>
>                 Key: SOLR-7332
>                 URL: https://issues.apache.org/jira/browse/SOLR-7332
>             Project: Solr
>          Issue Type: Sub-task
>          Components: SolrCloud
>            Reporter: Timothy Potter
>            Assignee: Timothy Potter
>         Attachments: SOLR-7332.patch
>
>
> See full discussion with Yonik and I in SOLR-6816.
> The TL;DR of that discussion is that we should initialize highest for each version bucket
to the MAX value of the {{__version__}} field in the index as early as possible, such as after
the first soft- or hard- commit. This will ensure that bulk adds where the docs don't exist
avoid an unnecessary lookup for a non-existent document in the index.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message