lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: how to setup to archive expired documents?
Date Sat, 17 Dec 2011 00:06:17 GMT

: So if we use some sort of weekly or daily sharding, there needs to be 
: some mechanism in place to dynamically add the new shard when the 
: current one fills up.  (Which would also ideally know where to put the 
: new shards on what server, etc.) Since SOLR does not implement that I 
: was thinking of just having a static set of shards.

You may want to consider taking a look at the ongoing work on improving 
"Solr Cloud" -- particularly the distributed indexing and shard failure 
logic.

My understanding (from past discussions, this may have changed w/o me 
realizing it) is that doc->shard mapping will nominally be a simple hash 
function on the uniqueKey, but that a plugin could customize that so that 
documents are sharded by date -- and then when you only need to query 
"recent" docs you could query those shards explicitly.  (no idea if things 
are stable enough yet for such a plugin to be written -- but the sooner 
someone tries the tackle it to solve their use case, the sooner people 
will be confident that the API is stable)

https://wiki.apache.org/solr/SolrCloud
https://issues.apache.org/jira/browse/SOLR-2358
https://svn.apache.org/viewvc/lucene/dev/branches/solrcloud/

-Hoss

Mime
View raw message