lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kelly, Frank" <frank.ke...@here.com>
Subject Re: Solr Heap Dump: Any suggestions on what to look for?
Date Thu, 09 Feb 2017 14:42:12 GMT
Thanks for the fast reply.

I think we¹re going to focus on using doc values.

You also said "facet on fewer fields² - how does one do that?

Thanks!

-Frank

 
Frank Kelly
Principal Software Engineer
 
HERE 
5 Wayside Rd, Burlington, MA 01803, USA
42° 29' 7" N 71° 11' 32" W
 
 <http://360.here.com/>     <https://www.twitter.com/here>
<https://www.facebook.com/here>
<https://www.linkedin.com/company/heremaps>
<https://www.instagram.com/here/>



On 2/9/17, 8:25 AM, "Markus Jelsma" <markus.jelsma@openindex.io> wrote:

>Hello - FieldCache is your problem. This can be solved in many ways but
>only one really beneficial: decrease number of documents, increase heap,
>facet on fewer fields, don't do function query on many fields. Or, of
>course, reindex with doc values. And you get a bonus, you can also
>drastically reduce your heap size.
>
>----Original message-----
>> From:Kelly, Frank <frank.kelly@here.com>
>> Sent: Thursday 9th February 2017 14:20
>> To: solr-user@lucene.apache.org
>> Subject: Solr Heap Dump: Any suggestions on what to look for?
>> 
>> Got a heap dump on an Out of Memory error.
>> Analyzing the dump now in Visual VM
>> 
> 
>> Seeing a lot of byte[] arrays (77% of our 8GB Heap) in
>> TreeMap$Entry FieldCacheImpl$SortedDocValues
>> We¹re considering switch over to DocValues but would rather be
>>definitive about the root cause before we experiment with DocValues and
>>require a reindex of our 200M document index
>> In each of our 4 data centers.
>> 
> 
>> Any suggestions on what I should look for in this heap dump to get a
>>definitive root cause?
>> 
> 
>> Cheers! 
>> 
> 
>> -Frank 
>> 
> 
>> 
> 
>> Frank Kelly 
>> Principal Software Engineer
>> HERE  
>> 5 Wayside Rd, Burlington, MA 01803, USA
>> 42° 29' 7" N 71° 11' 32" W
>>  
>><https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F360.he
>>re.com%2F&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798%7C6d4034cd72
>>254f72b85391feaea64919%7C1&sdata=bLudPbCB2aeqAFdvvZ%2BS5EYg37m%2BKPSTXYNY
>>Vz%2B1Eug%3D&reserved=0>
>>  
>><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.t
>>witter.com%2Fhere&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798%7C6d
>>4034cd72254f72b85391feaea64919%7C1&sdata=qxDdM94Mnlw02QHD8oFyzOHO3pNvyHeb
>>HvSvBz767hE%3D&reserved=0>
>>  
>><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.f
>>acebook.com%2Fhere&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798%7C6
>>d4034cd72254f72b85391feaea64919%7C1&sdata=uLkky5W5MPUXddFkr5LqDrzn0MRt0vB
>>kvnxuZhlxpqo%3D&reserved=0>
>>  
>><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.l
>>inkedin.com%2Fcompany%2Fheremaps&data=01%7C01%7C%7Ca5c3d440116b4ff1417508
>>d450ef1798%7C6d4034cd72254f72b85391feaea64919%7C1&sdata=fKBiSO7A1YPG4uRzy
>>WPqyYDpH1B9l%2FyUqCG2TzeLWgM%3D&reserved=0>
>>  
>><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.i
>>nstagram.com%2Fhere%2F&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798
>>%7C6d4034cd72254f72b85391feaea64919%7C1&sdata=4T41MNcQpb1jH7qzJrySt79MlVj
>>YwyYLG2lZXVioEkw%3D&reserved=0>


Mime
View raw message