lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley (JIRA)" <>
Subject [jira] [Commented] (SOLR-7332) Seed version buckets with max version from index
Date Wed, 22 Apr 2015 21:29:59 GMT


Yonik Seeley commented on SOLR-7332:

bq. Next, I tried increasing the number of reducers I was using to see how hard I could push
Solr and unfortunately, I ended up with 2 shards that had replicas that were out-of-sync with
their leader. 

Were there any recoveries or change of leaders during the run?
In a way, this is great that you saw this!  Only new adds should significantly narrow what
this could be.  Hopefully you'll be able to reproduce.

bq. can you think of a case where docs could be dropped with this new version bucket seeding

No... if we accidentally set the version too high, there are no correctness issues, just extra
If we accidentally set the version too low, then we can fail to drop repeated or reordered
updates.  But in your test run, this shouldn't be an issue since it's only adds.  Any old
repeats won't change the number of docs (and which docs) are in the index.

> Seed version buckets with max version from index
> ------------------------------------------------
>                 Key: SOLR-7332
>                 URL:
>             Project: Solr
>          Issue Type: Sub-task
>          Components: SolrCloud
>            Reporter: Timothy Potter
>            Assignee: Timothy Potter
>         Attachments: SOLR-7332.patch, SOLR-7332.patch, SOLR-7332.patch, SOLR-7332.patch,
> See full discussion with Yonik and I in SOLR-6816.
> The TL;DR of that discussion is that we should initialize highest for each version bucket
to the MAX value of the {{__version__}} field in the index as early as possible, such as after
the first soft- or hard- commit. This will ensure that bulk adds where the docs don't exist
avoid an unnecessary lookup for a non-existent document in the index.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message