lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wolfgang hoschek (JIRA)" <>
Subject [jira] [Commented] (SOLR-6126) MapReduce's GoLive script should support replicas
Date Mon, 02 Jun 2014 08:50:01 GMT


wolfgang hoschek commented on SOLR-6126:

[~dsmiley] It uses the --zk-host CLI options to fetch the solr URLs of each replica from zk
- see extractShardUrls(). This info gets passed via the Options.shardUrls parameter into the
go-live phase. In the go-live phase the segments of each shard are explicitly merged via a
separate REST merge request per replica into the corresponding replica. The result is that
each input segment is explicitly merged N times where N is the replication factor. Each such
merge reads from HDFS and writes to HDFS.

(BTW, I'll be unreachable on an transatlantic flight very soon)

> MapReduce's GoLive script should support replicas
> -------------------------------------------------
>                 Key: SOLR-6126
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: contrib - MapReduce
>            Reporter: David Smiley
> The GoLive feature of the MapReduce contrib module is pretty cool.  But a comment in
there indicates that it doesn't support replicas.  Every production SolrCloud setup I've seen
has had replicas!
> I wonder what is needed to support this.  For GoLive to work, it assumes a shared file
system (be it HDFS or whatever, like a SAN).  If perhaps the replicas in such a system read
from the very same network disk location, then all we'd need to do is send a commit() to replicas;

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message