lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Solr 5.2.1 on Java 8 GC
Date Sun, 01 May 2016 17:37:52 GMT
On 4/28/2016 9:43 AM, Nick Vasilyev wrote:
> I forgot to mention that the index is approximately 50 million docs split
> across 4 shards (replication factor 2) on 2 solr replicas.

Later in the thread, Jeff Wartes mentioned my wiki page for GC tuning.

https://wiki.apache.org/solr/ShawnHeisey#GC_Tuning_for_Solr

Oracle seems to be putting most of their recent work into the G1
collector.  They want to make it the default collector for Java 9.  I
don't know if this is going to happen, or it they will delay that change
until Java 10 ... but I think that the default *is* going to eventually
be G1.

Lucene strongly recommends against EVER using G1, but I have not noticed
any problems with it on Solr.  I have not seen any specific *reason*
that G1 is not recommended, at least with a 64-bit JVM.  I can only
remember seeing issues in Jira with G1 when the JVM was 32-bit -- which
has its own limitations, so I don't recommend it anyway.  If somebody
can point me to specific *OPEN* issues showing current problems with G1
on a 64-bit JVM, I will have an easier time believing the Lucene
assertion that it's a bad idea.  I have searched Jira and can't find
anything relevant.

The best GC results I've seen in testing Solr have been with the G1
collector.  I haven't done any testing for quite a while, and almost all
of the testing that I've done has been with 4.x versions, not 5.x.

Out of the box, Solr 5.0 and later uses GC tuning with the CMS collector
that looks a lot like the CMS config that I came up with.  This works
pretty well, but if the heap size gets big enough, especially 32GB or
larger, I suspect that it will start to have problems with GC performance.

You could give the settings I listed under "Current experiments" a try. 
You would do this by editing your solr.in.* script to comment out the
current GC tuning parameters and substituting the new set of
parameters.  This is a G1 config, and as I already mentioned, Lucene
recommends NEVER using G1.

For your specific situation, I think you should try setting the max heap
to 31GB instead of 32GB, so the pointer sizes are cut in half, without
making a huge difference in the total amount of memory available.

The chewiebug version of GCViewer is what I use to take the GC logfile
and see how Solr did with particular GC settings.  At my request, they
have incorporated some really nice statistical metrics into GCViewer,
but it's not available in the released versions -- you'll have to
compile it yourself until 1.35 comes out.

https://github.com/chewiebug/GCViewer/issues/139

I have also had some good luck with jHiccup.  That tool will inform you
of pauses that happen for *any* reason, not just because of GC.  My
experience is that GC is the only *major* cause of pauses.

Thanks,
Shawn


Mime
View raw message