lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Branham, Jeremy [HR]" <Jeremy.D.Bran...@sprint.com>
Subject RE: SOLR reindexing
Date Fri, 01 Mar 2013 21:03:11 GMT
Thanks Shawn.



Jeremy D. Branham
Performance Technologist II
Sprint University Performance Support
Fort Worth, TX | Office: +1 (972) 405-2970 | Mobile: +1 (817) 791-1627
http://JeremyBranham.Wordpress.com
http://www.linkedin.com/in/jeremybranham


-----Original Message-----
From: Shawn Heisey [mailto:solr@elyograg.org]
Sent: Friday, March 01, 2013 2:31 PM
To: solr-user@lucene.apache.org
Subject: Re: SOLR reindexing

On 3/1/2013 12:01 PM, Branham, Jeremy [HR] wrote:
> I've read this...
> http://stackoverflow.com/questions/5154093/solr-requests-time-out-duri
> ng-index-update-perhaps-replication-a-possible-solut
>
> [Using SOLR 1.4]
>
> We are doing intraday full re-index because we aren't sure if the document has been modified.
> Currently we are re-indexing to a second core then swapping to the original.
>
> Is this still the preferred method to avoid searcher delays?
>
> [We are planning an upgrade to SOLR 4 soon, if that makes any
> difference]

For full reindexes (DIH full-import), I use build cores, then swap them with the live cores.
 I don't do this for performance reasons, I do it because I want to continue making incremental
updates to the live cores while the rebuild is underway.  The rebuild takes four hours.

It's normal for searches to take longer while an index rebuild is happening.  The extreme
change that the stackoverflow user is seeing (queries going from 80 milliseconds to over 10000
milliseconds) suggests fundamental performance problems, possibly from an undersized server
that doesn't have enough RAM for the OS to cache the index files.  With regular full rebuilds
in another core, you'd actually want enough RAM to cache both the old index and the new one.

If you need to make visible updates to the existing index while building a new one, then your
current practice of building in another core and swapping is the only reasonable solution.
 I would probably stick to that model unless you're planning to move to SolrCloud, which works
very differently and doesn't have collection swapping available.

Thanks,
Shawn



________________________________

This e-mail may contain Sprint Nextel proprietary information intended for the sole use of
the recipient(s). Any use by others is prohibited. If you are not the intended recipient,
please contact the sender and delete all copies of the message.


Mime
View raw message