lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Commit takes 1 to 2 minutes, CPU usage affects other apps
Date Tue, 04 May 2010 01:24:04 GMT
On 5/3/10 9:06 AM, Markus Fischer wrote:
> Hi,
>
> we recently began having trouble with our Solr 1.4 instance. We've about
> 850k documents in the index which is about 1.2GB in size; the JVM which
> runs tomcat/solr (no other apps are deployed) has been given 2GB.
>
> We've a forum and run a process every minute which indexes the new
> messages. The number of messages updated are from 0 to 20 messages
> average. The commit takes about 1 or two minutes, but usually when it
> finished a few seconds later the next batch of documents is processed
> and the story starts again.
>
> So actually it's like Solr is running commits all day long and CPU usage
> ranges from 80% to 120%.
>
> This continuous CPU usage caused ill effects on other services running
> on the same machine.
>
> Our environment is being providing by a company purely using VMWare
> infrastructure, the Solr index itself is on an NSF for which we get some
> 33MB/s throughput.
>
> So, an easy solution would be to just put more resources into it, e.g. a
> separate machine. But before I make the decision I'd like to find out
> whether the app behaves properly under this circumstances or if its
> possible to shorten the commit time down to a few seconds so the CPU is
> not drained that long.
>
> thanks for any pointers,
>
> - Markus
>

That is certainly not a normal commit time for an index of that size.

Note that Solr 1.4 can have issues when working on NFS, but I don't know 
that it would have anything to do with this.

Are you using the simple lock factory rather than the default native 
lock factory? (as you should do when running on NFS)

-- 
- Mark

http://www.lucidimagination.com

Mime
View raw message