lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yao <...@ford.com>
Subject RE: Date faceting and memory leaks
Date Tue, 18 May 2010 01:24:11 GMT

Just to close the loop.
 
I was fooling around the all the cache setting trying to figure out my
problem, so the filterCache is set as part of the experiments. It did
not cause any memory issue in this case. After the date rounding
adjustment, I re-ran the query with 15 threads with 6000 request and got
1,500/minute throughput by only using a little more than 0.5 GB of Heap
Memory.
 
The hit ratio reported in Solr admin statistics page shows filterCache
has a hitratio of 0.99. with 103800 lookups and 103773 hits, I assume it
is 99%. 
 
Have a nice day.
 
-Yao

________________________________

From: Chris Hostetter-3 [via Lucene]
[mailto:ml-node+825052-1711725506-201948@n3.nabble.com] 
Sent: Monday, May 17, 2010 9:04 PM
To: Ge, Yao (Y.)
Subject: Re: Date faceting and memory leaks


: Cache settings: 
:   <filterCache class="solr.LRUCache" size="1512000"
initialSize="1512000" 
: autowarmCount="1280"/> 

that's a monster filterCache ...i can easly imagine it causing an OOM if

your heap is only 5G. 

: The date rounding suggest is a very good one, I will need to rerun the
test 
: and report back on the cache setting. I remember my filterCache hit
ratio is 
: around 0.7. I did use the tagged results for multi-select display of
facet 

a "hit ratio" or "0.7" ratio, or "0.7% hit rate"? ... with that many 
unique facet queries, i can't imaging you were getting a 70% hit rate.

I'm betting if you monitor that filterCache size and hit rate as you run

your test you'll see it just grow and grow until the OOM.  and if you 
analyze the heap dumps you'll probably see the cache hanging on to a ton

of DocSets that will never be used again. 

: values but in this case there is no fq in the load test request URL. 

I've never tested this, so i can't say for sure, but if it turns out
that 
the filterCache is not your problem, then perhaps there is soemthing
wonky 
with the filterquery exclusion code in cases like this -- where you 
explicilty exlucde a taged fq but that fq doesn't exist.  the qya to
rule 
it out would be to remove the exlcusion from your configs and test it
that 
way to see if the behavior is hte same. 


-Hoss 




________________________________

View message @
http://lucene.472066.n3.nabble.com/Date-faceting-and-memory-leaks-tp8243
72p825052.html 
To unsubscribe from Re: Date faceting and memory leaks, click here
< (link removed) 
WdlQGZvcmQuY29tfDgyNTAzOHwxNjYwNDQ2MTQ1> . 


-- 
View this message in context: http://lucene.472066.n3.nabble.com/Date-faceting-and-memory-leaks-tp824372p825086.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message