lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renaud Delbru <renaud.del...@deri.org>
Subject Re: Slow deleteById request
Date Tue, 01 Jul 2008 20:05:20 GMT
Hi Yonik,

We are not sending a commit with a delete. It happens when using the 
following command:
curl http://mydomain.net:8080/index/update -s -H 'Content-type:text/xml; 
charset=utf-8' -d "<delete><id>http://example.org/</id></delete>"
or using the SolrJ deleteById method (that does not execute a commit as 
far as I know).

The strange things is that it is not always reproduced. Ten or so delete 
requests will be executed fast (in few ms), then a batch of few delete 
requests will take 10, 20 or even 30 seconds.

By looking more precisely at the log, it seems that, in fact,  the 
delete request triggers the opening of a new searcher, with its 
auto-warming. On a large index (our case), I heard that it can take 
quite some time. Anyway, I am without a precise explanation about this 
problem.
This is not a big issue in our case, since it occurs for few requests 
and since other concurrent requests will be handled by the other searcher.

-- 
Renaud Delbru


Yonik Seeley wrote:
> That's very strange... are you sending a commit with the delete perhaps?
> If so, the whole request would block until a new searcher is registered.
>
> -Yonik
>
> On Tue, Jul 1, 2008 at 8:54 AM, Renaud Delbru <renaud.delbru@deri.org> wrote:
>   
>> Hi,
>>
>> We experience very slow delete, taking more than 10 seconds. A delete is
>> executed using deleteById (from Solrj or from curl), at the same time
>> documents are being added.
>> By looking at the log (below), it seems that a delete by ID request is only
>> executed during the next commit (done automatically every 1000 added
>> documents), and that the process (Solrj or curl) executing the deleteById
>> request is blocked until the commit is performed.
>>
>> Is it a normal behavior or a misconfiguration of our Solr server ?
>>
>> Thanks in advance for insights.
>>
>> [11:32:02.840]autowarming result for Searcher@19ba8886 main
>> [11:32:02.840]
>>  queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=512,evictions=0,size=512,cumulative_lookups=238825,cumulative_hits=202879,cumulative_hitratio=0.84,cumulative_inserts=36255,c
>> umulative_evictions=4289}
>> [11:32:02.840]autowarming Searcher@19ba8886 main from Searcher@1e6e2c86 main
>> [11:32:02.840]
>>  documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=2395306,cumulative_hits=1705483,cumulative_hitratio=0.71,cumulative_inserts=689823,cumulative_evictions=411577}
>> [11:32:02.840]autowarming result for Searcher@19ba8886 main
>> [11:32:02.840]
>>  documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=2395306,cumulative_hits=1705483,cumulative_hitratio=0.71,cumulative_inserts=689823,cumulative_evictions=411577}
>> [11:32:02.840]Registered new searcher Searcher@19ba8886 main
>> [11:32:02.840]{delete=[http://example.org/]} 0 14212
>> [11:32:02.840]webapp=/index path=/update params={wt=xml&version=2.2}
>> status=0 QTime=14212
>> [11:32:02.840]DirectUpdateHandler2 deleting and removing dups for 217 ids
>> [11:32:02.840]Closing Writer DirectUpdateHandler2
>> [11:32:02.842]Closing Searcher@1e6e2c86 main
>> [11:32:02.842]
>>  filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>> [11:32:02.842]
>>  queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=512,evictions=0,size=512,cumulative_lookups=238825,cumulative_hits=202879,cumulative_hitratio=0.84,cumulative_inserts=36255,cumulative_evictions=4289}
>> [11:32:02.842]
>>  documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,cumulative_lookups=2395306,cumulative_hits=1705483,cumulative_hitratio=0.71,cumulative_inserts=689823,cumulative_evictions=411577}
>> [11:32:02.894]Opening Searcher@7d33e3b7 DirectUpdateHandler2
>> [11:32:03.566]DirectUpdateHandler2 docs deleted=0
>> [11:32:03.566]Closing Searcher@7d33e3b7 DirectUpdateHandler2
>>
>> --
>> Renaud Delbru
>>
>>     

Mime
View raw message