lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr cache size information
Date Sun, 04 Dec 2011 02:48:02 GMT
See below:

On Thu, Dec 1, 2011 at 10:57 AM, elisabeth benoit
<elisaelisaelisa@gmail.com> wrote:
> Hello,
>
> If anybody can help, I'd like to confirm a few things about Solr's caches
> configuration.
>
> If I want to calculate cache size in memory relativly to cache size in
> solrconfig.xml
>
> For Document cache
>
> size in memory = size in solrconfig.xml * average size of all fields
> defined in fl parameter   ???

pretty much.

>
> For Filter cache
>
> size in memory = size in solrconfig.xml * WHAT (the size of an id) ??? (I
> don't use facet.enum method)
>

It Depends(tm). Solr tries to do the best thing here, depending upon
how many docs match the filter query. One method puts in a bitset for
each
entry, which is (maxDocs/8) bytes. maxDocs is reported on the admin/stats
page.

If the filter cache only hits a few documents, the size is smaller than that.

You can think of this cache as a map where the key is the
filter query (which is how they're re-used and how autowarm
works) and the value for each key is the bitset or list. The
size of the map is bounded by the size in solrconfig.xml.

> For Query result cache
>
> size in memory = size in solrconfig.xml * the size of an id ???
>
Pretty much. This is the maximum size, but each entry is
the query plus a list of IDs that's up to <queryResultWindowSize>
long. This cache is, by and large, the least of your worries.


>
> I would also like to know relation between solr's caches sizes and JVM max
> size?

Don't quite know what you're asking for here. There's nothing automatic
that's sensitive to whether the JVM memory limits are about to be exceeded.
If the caches get too big, OOMs happen.

>
> If anyone has an answer or a link for further reading to suggest, it would
> be greatly appreciated.
>
There's some information here: http://wiki.apache.org/solr/SolrCaching, but
it often comes down to "try your app and monitor"....

Here's a work-in-progress that Grant is working on, be aware that it's
for trunk, not 3x.
http://java.dzone.com/news/estimating-memory-and-storage


Best
Erick

> Thanks,
> Elisabeth

Mime
View raw message