lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <cbenn...@job.com>
Subject RE: Commit takes 1 to 2 minutes, CPU usage affects other apps
Date Tue, 04 May 2010 12:43:49 GMT
Hi,

This could also be caused by performing an optimize after the commit, or it
could be caused by auto warming the caches, or a combination of both.

If you are using the Data Import Handler the default for a delta import is
commit and optimize, which caused us a similar problem except we were
optimizing a 7 million document, 23Gb index with every delta import which
was taking over 10 minutes. As soon as we added optimize=false to the
command updates took a few seconds. You can always add separate calls to
perform the optimize when it's convenient for you.

To see if the problem is auto warming take a look at the warm up time for
the searcher. If this is the cause you will need to consider lowering the
autowarmCount for your caches. 


Colin.

> -----Original Message-----
> From: Markus Fischer [mailto:markus@fischer.name]
> Sent: Tuesday, May 04, 2010 6:22 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Commit takes 1 to 2 minutes, CPU usage affects other apps
> 
> On 04.05.2010 11:01, Peter Sturge wrote:
> > It might be worth checking the VMWare environment - if you're using
> the
> > VMWare scsi vmdk and it's shared across multiple VMs and there's a
> lot of
> > disk contention (i.e. multiple VMs are all busy reading/writing
> to/from the
> > same disk channel), this can really slow down I/O operations.
> 
> Ok, thanks, I'll try to get the information from my hoster.
> 
> I noticed that the commiting seems to be constant in time: it doesn't
> matter whether I'm updating only one document or 50 (usually it won't
> be
> more). Maybe these numbers are so low anyway to cause any real impact
> ...
> 
> - Markus





Mime
View raw message