lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: SolrCloud Scale Struggle
Date Mon, 11 Aug 2014 13:11:25 GMT
On 8/10/2014 11:07 PM, anand.mahajan wrote:
> Thank you for your suggestions. With the autoCommit (every 10 mins) and
> softCommit (every 10 secs) frequencies reduced things work much better now.
> The CPU usages has gone down considerably too (by about 60%) and the
> read/write throughput is showing considerable improvements too. 
> 
> There are a certain shards that are giving poor response times - these have
> over 10M listings - I guess this is due to the fact that these are starving
> for RAM? Would it help if I split these up in smaller shards, but with the
> existing set of hardware? (I cannot allocate more machines to the cloud as
> yet)

Memory requirements are actually likely to go *up* a little bit with
more shards on the same hardware, not down.  The ideal RAM setup is to
have enough RAM in the machine to equal or exceed the sum of your max
Solr heap and the size of all the index data on that machine.  This
allows the operating system to load the entire index into the disk
cache.  Disks are *slow*, if the OS can pull the data it needs out of
RAM (the operating system disk cache), that becomes *very* fast.

If you have enough RAM to load at least two thirds of the index size,
performance is likely to also be very good, but 100% of the index is better.

If at all possible, put more RAM in the machine.

Thanks,
Shawn


Mime
View raw message