lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marship <mars...@126.com>
Subject Re:Re: How to speed up solr search speed
Date Sat, 17 Jul 2010 16:36:56 GMT
Hi. Shawn.
My indexes are smaller than yours. I only store "id" + "type" in indexes so each "core" index
is about 1 - 1.5GB on disk.
I don't have so many servers/VPS as you have. In my option, my problem is not CPU. If possible,
I prefer to add more memory to fit indexes in my server. At least at memory is cheaper. And
I saw lots of my CPU time are wasted because no program can fullly use it. 

Is there a way to tell solr to load all indexes into memory? like memory directory in lucene.
That would be breezing fast
Btw, how do you get that status page? 

Thanks.
Regards.
Scott



在2010-07-17 23:38:58,"Shawn Heisey" <solr@elyograg.org> 写道:
>  On 7/17/2010 3:28 AM, marship wrote:
>> Hi. Peter and All.
>> I merged my indexes today. Now each index stores 10M document. Now I only have 10
solr cores.
>> And I used
>>
>> java -Xmx1g -jar -server start.jar
>> to start the jetty server.
>
>How big are the indexes on each of those cores? You can easily get this 
>info from a URL like this (assuming the bundled Jetty and its standard 
>port):
>
>http://hostname:8983/solr/corename/admin/replication/index.jsp
>
>If your server only has 4GB of RAM, low memory is almost guaranteed to 
>be the true problem. With low ram levels, the disk cache is nearly 
>useless, and high disk I/O is the symptom.
>
>My system runs as virtual machines. I've got six static indexes each a 
>little over 12GB in size (7 million rows) and an incremental index that 
>gets to about 700MB (300,000 rows). I've only got one active index core 
>per virtual machine, except when doing a full reindex, which is rare. 
>Each static VM is allocated 2 CPUs and 9GB of memory, each incremental 
>has 2 CPUs and 3GB of memory. As I'm not using VMware, the memory is not 
>oversubscribed. There is a slight oversubscription of CPUs, but I've 
>never seen a CPU load problem. I've got dedicated VMs for load balancing 
>and for the brokers.
>
>With a max heap of 1.5GB, that leaves over 7GB of RAM to act as disk 
>cache for a 12GB index. My statistics show that each of my two broker 
>cores has 185000 queries under its belt, with an average query time of 
>about 185 milliseconds. If I had enough memory to fit the entire 12GB 
>index into RAM, I'm sure my query times would be MUCH smaller.
>
>Here's a screenshot of the status page that aggregates my Solr statistics:
>
>http://www.flickr.com/photos/52107710@N05/4801491979/sizes/l/
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message