lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shawn Heisey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5940) change index backwards compatibility policy.
Date Fri, 19 Sep 2014 15:19:33 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14140723#comment-14140723
] 

Shawn Heisey commented on LUCENE-5940:
--------------------------------------

{quote}
also, "reindexing" comes in 2 distinct flavors:
* cold reindexing - rm -rf the index dir, re feed
** requires 2x hardware or downtime
* live reindexing - change config, restart system, re feed all docs, change is "live" once
all docs have been reindexed
** obviously a good idea to snapshot any previous index and config so you can restore later
on error
** minimal downtime (just restart)
** minimal search interruption (some queries related to the change may not match old documents
until reindex is complete)
** old content can be replaced slowly over time to receive full functionality
{quote}

I use Solr.  My reindexing method is actually a combination of the two you've mentioned. 
For every shard, I have a live core and a build core.  When a reindex is required, I start
importing from my database into the build cores.  In the meantime, the live cores are still
being updated once a minute with new data and deletes.  When the full import is done, I apply
all relevant changes to the build cores, then swap them with the live cores.  Once that copy
of my index is rebuilt, I re-enable it so that the load balancer can use it again.


> change index backwards compatibility policy.
> --------------------------------------------
>
>                 Key: LUCENE-5940
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5940
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>
> Currently, our index backwards compatibility is unmanageable. The length of time in which
we must support old indexes is simply too long.
> The index back compat works like this: everyone wants it, but there are frequently bugs,
and when push comes to shove, its not a very sexy thing to work on/fix, so its hard to get
any help.
> Currently our back compat "promise" is just a broken promise, because we cannot actually
guarantee it for these reasons.
> I propose we scale back the length of time for which we must support old indexes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message