lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Core failure when a lot of processes are indexing
Date Sun, 04 May 2014 13:23:58 GMT
On 5/4/2014 6:06 AM, Hakim Benoudjit wrote:
> I have a lot of scripts running *concurrently *& indexing on the same index
> (*collection*). This throws the following error:
> 
> *SolrCore 'collection2' is not available due to init failure: Error opening
> new searcher</str><str name="trace">org.apache.solr.common.SolrException:
> SolrCore 'collection2' is not available due to init failure: Error opening
> new searcher*
> 
> Which from what I've understood means that the index has been *corrupted*.

This can mean a *lot* of things.  Index corruption is one of the last
things I would suspect.  It can happen, but it is highly unlikely.

You would need to include the full error message (which could be more
than one line) and the java stacktrace (which will be MANY lines). This
may mean you need to find the logfile, not something that shows up in
the admin UI.  We'd also need to know the Solr version and as many other
details about your server, java, Solr startup, and index as you can
include.  The config and schema would be great to have.  Note that the
mailing list does not deal well with attachments, so anything like
screenshots and configs will generally need to be hosted elsewhere and
linked.

> So what is the solution that you can suggest me to use? Active Messagin
> (ActiveMQ or RabbitMQ) maybe?
> 
> *PS: *
> -- By adding a *sleep() *in each of these scripts, the index managed to not
> be corrupted, that's why I'm suspecting that the number of concurrent
> request (*update*) received by solr, are causing this error.

Solr is fully threadsafe.  It can handle multiple concurrent requests.
There was a relatively recent problem with multiple concurrent indexing
requests against Solr when running in SolrCloud mode with many shards,
but as far as I know this did not lead to problems starting a searcher,
it caused indexing to hang and not proceed.

Thanks,
Shawn


Mime
View raw message