lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Connor" <ian.con...@gmail.com>
Subject Re: estimating memory needed for solr instances...
Date Wed, 09 Jul 2008 20:15:30 GMT
There was a thread a while ago, that suggested just need to factor in
the index's total size (Mike Klaas I think was the author). It was
suggested having the RAM is enough and the OS will cache the files as
needed to give you the performance boost needed.

If I misread the thread, please chime in - but it seems having enough
RAM is the key to performance.

On Wed, Jul 9, 2008 at 3:00 AM, Preetam Rao <blogathan.rao@gmail.com> wrote:
> Hi,
>
> Since we plan to share the same box among multiple solr instances on a 16gb
> RAM multi core box, Need to estimate how much memory we need for our
> application.
>
> The index size is on disk  2.4G with close to 3 million documents. The plan
> is to use dismax query with some fqs.
> Since we do not sort the results, the sort will be by score which eliminates
> the option "userFiterFprSortedQuerries".
> Thus assuming all q's will use query result cache and all fqs will use
> filter caches the below is what i am thinking.
>
> I would like to know how to relate the index size on disk to its memory size
> ?
> Would it be safe to assume gven the disk size of 2.4g, that we can have ram
> size for whole index plus 1g for any other overhead plus the cache size
> which comes to 150MB  (calculation below). Thus making it around 4g.
>
> cache size calculation -
> --------------------------------
> query result cache - size = 50K;
> since we paginate the results and each page has 10 items and assuming each
> user will at the max see 3 pages, per query
> we will set queryResultWindowSize to 30. Assuming this, for 50k querries we
> will use up 50000* 30 bits = 187K asuming results are stored in bitset.
>
> we use few common fqs, lets say 200. Assuming each returns around 30k
> documents, it adds to 200 * 30000 bits  = 750K.
>
> If we use document cache of size 20K, assuming each document size is around
> 5k at the max, it will take up 20000 * 5= 100MB.
>
> Thus we can increase the cache more drastically and still it will use up
> only 150MB or less.
>
> Is this reasoning on cache's correct ?
>
> Thanks
> Preetam
>



-- 
Regards,

Ian Connor
82 Fellsway W #2
Somerville, MA 02145
Direct Line: +1 (978) 6333372
Call Center Phone: +1 (714) 239 3875 (24 hrs)
Mobile Phone: +1 (312) 218 3209
Fax: +1(770) 818 5697
Suisse Phone: +41 (0) 22 548 1664
Skype: ian.connor

Mime
View raw message