lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Jelsma <markus.jel...@openindex.io>
Subject RE: Slowly running OOM due to Query instances?!
Date Fri, 07 Jul 2017 13:41:58 GMT
Hello,

Sorry, i forgot to mention we already run 6.6. 

Now i am looking at the sampler again on a freshly restarted instance. This node has almost
6000 uncollectable TermQuery instances at this moment. There are just 3.4 queries per second
on this node. The final query is complex but i cannot understand why so many TermQuery. Is
this even normal? I haven't seen this before, but i also have to say i never looked at this
specific thing before.

Thanks,
Markus
 
 
-----Original message-----
> From:Erik Hatcher <erik.hatcher@gmail.com>
> Sent: Friday 7th July 2017 15:11
> To: solr-user@lucene.apache.org
> Subject: Re: Slowly running OOM due to Query instances?!
> 
> With generated Query’s, one has to be really careful with .equals and .hashCode implementations.
 That may not be applicable here, but something that has bitten me with caching.   Note that
there were fixes made in Solr 6.6 with PayloadScoreQuery in this regard. See LUCENE-7808 and
LUCENE-7481
> 
> 	Erik
> 
> 
> > On Jul 7, 2017, at 7:01 AM, Markus Jelsma <markus.jelsma@openindex.io> wrote:
> > 
> > Hello,
> > 
> > This morning i spotted our QTime suddenly go up. This has been going on for a few
hours by now and coincides with a serious increase in heap consumption. No node ran out of
memory so far but either that is going to happen soon, or the nodes become unusable in another
manner.
> > 
> > I restarted one of the Solr instances and launched VisualVM at it, and some other
nodes that use to much heap. Starting the memory sampler, something was obvious straight away.
> > 
> > The nodes consuming too much heap all have a serious amount of *Query, and BooleanClause
instances, PayloadScoreQuery, TermQuery, BoostQuery, BooleanQuery, SpanTermQuery and so forth.
Lots of Builder and Term instances too, very distinct from the node that was just freshly
restarted.
> > 
> > Another peculiarity, some nodes have exactly 65536 instances of TermQuery and/or
BoostQuery, probably unrelated but not something i would have expected to see anyway.
> > 
> > So, what's up? We do have a custom query parser extending EdismaxQParser, it transliterates
dates and creates payload and span queries. I may be doing something wrong but i don't know,
i have made and used a variety of QParsers, for many years but this is new. Any hints on where
to look, what to watch out for? 
> > 
> > Many thanks!
> > Markus
> > 
> > Xmx 800m, 8 GB RAM, SSD
> > 2 shards, three replica's
> > replica size ~17 GB, 2.2 million docs/replica
> 
> 

Mime
View raw message