lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: SOLR reindexing
Date Fri, 01 Mar 2013 20:31:05 GMT
On 3/1/2013 12:01 PM, Branham, Jeremy [HR] wrote:
> I've read this...
> http://stackoverflow.com/questions/5154093/solr-requests-time-out-during-index-update-perhaps-replication-a-possible-solut
>
> [Using SOLR 1.4]
>
> We are doing intraday full re-index because we aren't sure if the document has been modified.
> Currently we are re-indexing to a second core then swapping to the original.
>
> Is this still the preferred method to avoid searcher delays?
>
> [We are planning an upgrade to SOLR 4 soon, if that makes any difference]

For full reindexes (DIH full-import), I use build cores, then swap them 
with the live cores.  I don't do this for performance reasons, I do it 
because I want to continue making incremental updates to the live cores 
while the rebuild is underway.  The rebuild takes four hours.

It's normal for searches to take longer while an index rebuild is 
happening.  The extreme change that the stackoverflow user is seeing 
(queries going from 80 milliseconds to over 10000 milliseconds) suggests 
fundamental performance problems, possibly from an undersized server 
that doesn't have enough RAM for the OS to cache the index files.  With 
regular full rebuilds in another core, you'd actually want enough RAM to 
cache both the old index and the new one.

If you need to make visible updates to the existing index while building 
a new one, then your current practice of building in another core and 
swapping is the only reasonable solution.  I would probably stick to 
that model unless you're planning to move to SolrCloud, which works very 
differently and doesn't have collection swapping available.

Thanks,
Shawn


Mime
View raw message