lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Solr maxIndexingThreads
Date Sun, 05 Jan 2014 20:59:09 GMT
Hi,

On Sun, Jan 5, 2014 at 9:59 AM, Saumitra Srivastav <
saumitra.srivastav7@gmail.com> wrote:

> I have a solr cluster with 8 server(4 shards with one replica for each). I
> have 80 client threads indexing to this cluster. Client is running on a
> different machine. I am trying to figure out optimal number of indexing
> threads.
>

You need to take into account a few more things:
* the number of CPU cores
* whether you are CPU or disk or network or memory-bound

Use a tool like SPM for Solr (see
http://sematext.com/spm/solr-performance-monitoring/ ) to gain insight into
various Solr, JVM, and OS metrics to understand what your bottleneck is and
how things change when you vary different parameters (e.g. soft commit
frequency, JVM heap size, ramBufferMemoryMB, the number of indexing threads
you are mentioning, etc.).

If you are using SolrCloud use the new CloudSolrServer and send docs in
batches.

btw. re:
"The maximum number of simultaneous threads that may be indexing documents
at once in IndexWriter; if more than this many threads arrive they will wait
for others to finish. Default in Solr/Lucene is 8. "

IndexWriter is a Lucene-level object. 1 IndexWriter writes to 1 Lucene
index.  1 Solr core == 1 Lucene index.

So another factor to consider is the optimal number of shards and replicas,
not just indexing threads.

Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/



>
> Now, solrconfig.xml have a config for *maxIndexingThreads*:
>
> "The maximum number of simultaneous threads that may be indexing documents
> at once in IndexWriter; if more than this many threads arrive they will
> wait
> for others to finish. Default in Solr/Lucene is 8. "
>
> I want to know whether this configuration is per solr instance or per
> core(or collection).
>
>
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-maxIndexingThreads-tp4109604.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message