lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-8586) Implement hash over all documents to check for shard synchronization
Date Mon, 08 Feb 2016 13:12:40 GMT

    [ https://issues.apache.org/jira/browse/SOLR-8586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15136924#comment-15136924
] 

Joel Bernstein commented on SOLR-8586:
--------------------------------------

Now that this is in place it may make sense to combine this with Streaming. The first thing
I see is to compare hashes between the shards and if there is a difference use the ComplementStream
to determine which id's are missing. The missing id's could then be automatically fetched
from the source and re-indexed. There could be a DaemonStream that lives inside the collection
that performs this check periodically. This could also sort out a situation where non of the
shards have the complete truth. 

> Implement hash over all documents to check for shard synchronization
> --------------------------------------------------------------------
>
>                 Key: SOLR-8586
>                 URL: https://issues.apache.org/jira/browse/SOLR-8586
>             Project: Solr
>          Issue Type: Improvement
>          Components: SolrCloud
>            Reporter: Yonik Seeley
>             Fix For: 5.5, Trunk
>
>         Attachments: SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch, SOLR-8586.patch
>
>
> An order-independent hash across all of the versions in the index should suffice.  The
hash itself is pretty easy, but we need to figure out when/where to do this check (for example,
I think PeerSync is currently used in multiple contexts and this check would perhaps not be
appropriate for all PeerSync calls?)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message