lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Memory leak in Solr
Date Fri, 02 Dec 2016 23:49:02 GMT
On 12/2/2016 12:01 PM, S G wrote:
> This post shows some stats on Solr which indicate that there might be a
> memory leak in there.
>
> http://stackoverflow.com/questions/40939166/is-this-a-memory-leak-in-solr
>
> Can someone please help to debug this?
> It might be a very good step in making Solr stable if we can fix this.

+1 to what Walter said.

I replied earlier on the stackoverflow question.

FYI -- your 95th percentile request time of about 16 milliseconds is NOT
something that I would characterize as "very high."  I would *love* to
have statistics that good.

Even your 99th percentile request time is not much more than a full
second.  If a search takes a couple of seconds, most users will not
really care, and some might not even notice.  It's when a large
percentage of queries start taking several seconds that complaints start
coming in.  On your system, 99 percent of your queries are completing in
1.3 seconds or less, and 95 percent of them are less than 17
milliseconds.  That sounds quite good to me.

In my experience, the time it takes for the browser to receive the
search result page and render it is a significant part of the total time
to see results, and often dwarfs the time spent getting info from Solr.

Here's some numbers from Solr in my organization:

requests:               4102054
errors:                 364894
timeouts:               49
totalTime:              799446287.45041
avgRequestsPerSecond:   1.2375565828793849
5minRateReqsPerSecond:  0.8444329508327961
15minRateReqsPerSecond: 0.8631197328073346
avgTimePerRequest:      194.88926460997587
medianRequestTime:      20.8566605
75thPcRequestTime:      85.51328849999999
95thPcRequestTime:      2202.277466549999
99thPcRequestTime:      5280.375381280002
999thPcRequestTime:     6866.020122961001

The numbers above come from a distributed index that contains 167
million documents and takes up about 200GB of disk space across two
machines.

requests:               192683
errors:                 124
timeouts:               0
totalTime:              199380421.985073
avgRequestsPerSecond    0.042222722771354554
5minRateReqsPerSecon    0.00800545427600684
15minRateReqsPerSecond: 0.017521222412364163
avgTimePerRequest:      1034.7587591280653
medianRequestTime:      541.591858
75thPcRequestTime:      1683.83246125
95thPcRequestTime:      5644.542019949997
99thPcRequestTime:      9445.592394760004
999thPcRequestTime:     14602.166640771007

These numbers are from an index with about 394 million documents, taking
up nearly 500GB of disk space.  This index is also distributed on
multiple machines.

Are you experiencing any problems other than what you perceive as slow
queries?  I asked some other questions on stackoverflow.  In particular,
I'd like to know the total memory on the server, the total number of
documents (maxDoc and numDoc) you're handling with this server, as well
as the total index size.  What do your queries look like?  What version
and vendor of Java are you using?  Can you share your config/schema?

A memory leak is very unlikely, unless your Java or your operating
system is broken.  I can't say for sure that it's not happening, but
it's just not something we see around here.

Here's what I have collected on performance issues in Solr.  This page
does mostly concern itself with memory, though it touches briefly on
other topics:

https://wiki.apache.org/solr/SolrPerformanceProblems

Thanks,
Shawn


Mime
View raw message