lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Chen <Tim.C...@sbs.com.au>
Subject RE: Solr Cloud with 5 servers cluster failed due to Leader out of memory
Date Fri, 05 Aug 2016 07:41:16 GMT
Thanks Guys. Very very helpful.

I will probably look at consolidate 4 Solr servers into 2 bigger/better server - it gives
more memory, and it cut down the replica the Leader needs to manage.

Also, I may look into write a script to monitor the tomcat log and if there is OOM, kill tomcat,
then restart it. A bit dirty, but may work for a short term.

I don't know too much about how documents indexed, and how to save memory from that. Will
probably work with a developer on this as well.

Many Thanks guys.

Cheers,
Tim

-----Original Message-----
From: Shawn Heisey [mailto:apache@elyograg.org]
Sent: Friday, 5 August 2016 4:55 PM
To: solr-user@lucene.apache.org
Subject: Re: Solr Cloud with 5 servers cluster failed due to Leader out of memory

On 8/4/2016 8:14 PM, Tim Chen wrote:
> Couple of thoughts: 1, If Leader goes down, it should just go down,
> like dead down, so other servers can do the election and choose the
> new leader. This at least avoids bringing down the whole cluster. Am I
> right?

Supplementing what Erick told you:

When a typical Java program throws OutOfMemoryError, program behavior is completely unpredictable.
 There are programming techniques that can be used so that behavior IS predictable, but writing
that code can be challenging.

Solr 5.x and 6.x, when they are started on a UNIX/Linux system, use a Java option to execute
a script when OutOfMemoryError happens.  This script kills Solr completely.  We are working
on adding this capability when running on Windows.

> 2, Apparently we should not pushing too many documents to Solr, how do
> you guys handle this? Set a limit somewhere?

There are exactly two ways to deal with OOME problems: Increase the heap or reduce Solr's
memory requirements.  The number of documents you push to Solr is unlikely to have a large
effect on the amount of memory that Solr requires.  Here's some information on this topic:

https://wiki.apache.org/solr/SolrPerformanceProblems#Java_Heap

Thanks,
Shawn



[Premiere League Starts Saturday 13 August 9.30pm on SBS]<http://theworldgame.sbs.com.au/>
Mime
View raw message