lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <>
Subject [jira] [Commented] (LUCENE-5940) change index backwards compatibility policy.
Date Fri, 19 Sep 2014 15:19:33 GMT


Shawn Heisey commented on LUCENE-5940:

also, "reindexing" comes in 2 distinct flavors:
* cold reindexing - rm -rf the index dir, re feed
** requires 2x hardware or downtime
* live reindexing - change config, restart system, re feed all docs, change is "live" once
all docs have been reindexed
** obviously a good idea to snapshot any previous index and config so you can restore later
on error
** minimal downtime (just restart)
** minimal search interruption (some queries related to the change may not match old documents
until reindex is complete)
** old content can be replaced slowly over time to receive full functionality

I use Solr.  My reindexing method is actually a combination of the two you've mentioned. 
For every shard, I have a live core and a build core.  When a reindex is required, I start
importing from my database into the build cores.  In the meantime, the live cores are still
being updated once a minute with new data and deletes.  When the full import is done, I apply
all relevant changes to the build cores, then swap them with the live cores.  Once that copy
of my index is rebuilt, I re-enable it so that the load balancer can use it again.

> change index backwards compatibility policy.
> --------------------------------------------
>                 Key: LUCENE-5940
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
> Currently, our index backwards compatibility is unmanageable. The length of time in which
we must support old indexes is simply too long.
> The index back compat works like this: everyone wants it, but there are frequently bugs,
and when push comes to shove, its not a very sexy thing to work on/fix, so its hard to get
any help.
> Currently our back compat "promise" is just a broken promise, because we cannot actually
guarantee it for these reasons.
> I propose we scale back the length of time for which we must support old indexes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message