lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Jelsma <markus.jel...@openindex.io>
Subject RE: Solr Heap Dump: Any suggestions on what to look for?
Date Thu, 09 Feb 2017 14:53:30 GMT
 
-----Original message-----
> From:Kelly, Frank <frank.kelly@here.com>
> Sent: Thursday 9th February 2017 15:42
> To: solr-user@lucene.apache.org
> Subject: Re: Solr Heap Dump: Any suggestions on what to look for?
> 
> Thanks for the fast reply.
> 
> I think we¹re going to focus on using doc values.
> 
> You also said "facet on fewer fields² - how does one do that?

Well, actually, you don't :) Or not meet functional requirements. 

> 
> Thanks!
> 
> -Frank
> 
>  
> Frank Kelly
> Principal Software Engineer
>  
> HERE 
> 5 Wayside Rd, Burlington, MA 01803, USA
> 42° 29' 7" N 71° 11' 32" W
>  
>  <http://360.here.com/>     <https://www.twitter.com/here>
> <https://www.facebook.com/here>
> <https://www.linkedin.com/company/heremaps>
> <https://www.instagram.com/here/>
> 
> 
> 
> On 2/9/17, 8:25 AM, "Markus Jelsma" <markus.jelsma@openindex.io> wrote:
> 
> >Hello - FieldCache is your problem. This can be solved in many ways but
> >only one really beneficial: decrease number of documents, increase heap,
> >facet on fewer fields, don't do function query on many fields. Or, of
> >course, reindex with doc values. And you get a bonus, you can also
> >drastically reduce your heap size.
> >
> >----Original message-----
> >> From:Kelly, Frank <frank.kelly@here.com>
> >> Sent: Thursday 9th February 2017 14:20
> >> To: solr-user@lucene.apache.org
> >> Subject: Solr Heap Dump: Any suggestions on what to look for?
> >> 
> >> Got a heap dump on an Out of Memory error.
> >> Analyzing the dump now in Visual VM
> >> 
> > 
> >> Seeing a lot of byte[] arrays (77% of our 8GB Heap) in
> >> TreeMap$Entry FieldCacheImpl$SortedDocValues
> >> We¹re considering switch over to DocValues but would rather be
> >>definitive about the root cause before we experiment with DocValues and
> >>require a reindex of our 200M document index
> >> In each of our 4 data centers.
> >> 
> > 
> >> Any suggestions on what I should look for in this heap dump to get a
> >>definitive root cause?
> >> 
> > 
> >> Cheers! 
> >> 
> > 
> >> -Frank 
> >> 
> > 
> >> 
> > 
> >> Frank Kelly 
> >> Principal Software Engineer
> >> HERE  
> >> 5 Wayside Rd, Burlington, MA 01803, USA
> >> 42° 29' 7" N 71° 11' 32" W
> >>  
> >><https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F360.he
> >>re.com%2F&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798%7C6d4034cd72
> >>254f72b85391feaea64919%7C1&sdata=bLudPbCB2aeqAFdvvZ%2BS5EYg37m%2BKPSTXYNY
> >>Vz%2B1Eug%3D&reserved=0>
> >>  
> >><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.t
> >>witter.com%2Fhere&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798%7C6d
> >>4034cd72254f72b85391feaea64919%7C1&sdata=qxDdM94Mnlw02QHD8oFyzOHO3pNvyHeb
> >>HvSvBz767hE%3D&reserved=0>
> >>  
> >><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.f
> >>acebook.com%2Fhere&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798%7C6
> >>d4034cd72254f72b85391feaea64919%7C1&sdata=uLkky5W5MPUXddFkr5LqDrzn0MRt0vB
> >>kvnxuZhlxpqo%3D&reserved=0>
> >>  
> >><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.l
> >>inkedin.com%2Fcompany%2Fheremaps&data=01%7C01%7C%7Ca5c3d440116b4ff1417508
> >>d450ef1798%7C6d4034cd72254f72b85391feaea64919%7C1&sdata=fKBiSO7A1YPG4uRzy
> >>WPqyYDpH1B9l%2FyUqCG2TzeLWgM%3D&reserved=0>
> >>  
> >><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.i
> >>nstagram.com%2Fhere%2F&data=01%7C01%7C%7Ca5c3d440116b4ff1417508d450ef1798
> >>%7C6d4034cd72254f72b85391feaea64919%7C1&sdata=4T41MNcQpb1jH7qzJrySt79MlVj
> >>YwyYLG2lZXVioEkw%3D&reserved=0>
> 
> 

Mime
View raw message