lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: How to speed up solr search speed
Date Sat, 17 Jul 2010 15:38:58 GMT
  On 7/17/2010 3:28 AM, marship wrote:
> Hi. Peter and All.
> I merged my indexes today. Now each index stores 10M document. Now I only have 10 solr
cores.
> And I used
>
> java -Xmx1g -jar -server start.jar
> to start the jetty server.

How big are the indexes on each of those cores? You can easily get this 
info from a URL like this (assuming the bundled Jetty and its standard 
port):

http://hostname:8983/solr/corename/admin/replication/index.jsp

If your server only has 4GB of RAM, low memory is almost guaranteed to 
be the true problem. With low ram levels, the disk cache is nearly 
useless, and high disk I/O is the symptom.

My system runs as virtual machines. I've got six static indexes each a 
little over 12GB in size (7 million rows) and an incremental index that 
gets to about 700MB (300,000 rows). I've only got one active index core 
per virtual machine, except when doing a full reindex, which is rare. 
Each static VM is allocated 2 CPUs and 9GB of memory, each incremental 
has 2 CPUs and 3GB of memory. As I'm not using VMware, the memory is not 
oversubscribed. There is a slight oversubscription of CPUs, but I've 
never seen a CPU load problem. I've got dedicated VMs for load balancing 
and for the brokers.

With a max heap of 1.5GB, that leaves over 7GB of RAM to act as disk 
cache for a 12GB index. My statistics show that each of my two broker 
cores has 185000 queries under its belt, with an average query time of 
about 185 milliseconds. If I had enough memory to fit the entire 12GB 
index into RAM, I'm sure my query times would be MUCH smaller.

Here's a screenshot of the status page that aggregates my Solr statistics:

http://www.flickr.com/photos/52107710@N05/4801491979/sizes/l/


Mime
View raw message