lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From adfel70 <adfe...@gmail.com>
Subject Re: severe problems with soft and hard commits in a large index
Date Wed, 06 May 2015 10:03:59 GMT
1. yes, I'm sure that pauses are due to GCs. I monitor the cluster and
receive continuously metric from system and from java process.
I see clearly that when soft commit is triggered, major GCs start occurring
(sometimes reocuuring on the same process) and latency rises.
I use CMS GC and jdk 1.7.75

2. My previous post was about another use case, but nevertheless I have
configured docvalues in the faceted fields.


Toke Eskildsen wrote
> On Wed, 2015-05-06 at 00:58 -0700, adfel70 wrote:
>> each shard has around 200 million docs. size of each shard is 250GB.
>> this runs on 12 machines. each machine has 4 SSD disks and 4 solr
>> processes.
>> each process has 28GB heap.  each machine has 196GB RAM.
> 
> [...]
> 
>> 1. heavy GCs when soft commit is performed (methods 1,2) or when
>> hardcommit
>> opensearcher=true is performed. these GCs cause heavy latency (average
>> latency is 3 secs. latency during the problem is 80secs)
> 
> Sanity check: Are you sure the pauses are due to garbage collection?
> 
> You have a fairly large heap and judging from your previous post
> "problem with facets  - out of memory exception", you are doing
> non-trivial faceting. Are you using DocValues, as Marc suggested?
> 
> 
> - Toke Eskildsen, State and University Library, Denmark





--
View this message in context: http://lucene.472066.n3.nabble.com/severe-problems-with-soft-and-hard-commits-in-a-large-index-tp4204068p4204088.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message