lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Britske <gbr...@gmail.com>
Subject Re: big discrepancy between elapsedtime and qtime although enableLazyFieldLoading= true
Date Wed, 30 Jul 2008 15:49:12 GMT

Hi Fuad, 


Funtick wrote:
> 
> 
> Britske wrote:
>> 
>> When performing these queries I notice a big difference between qTime
>> (which is mostly in the 15-30 ms range due to caching) and total time
>> taken to return the response (measured through SolrJ's elapsedTime),
>> which takes between 500-1600 ms. 
>> Documents have a lot of stored fields (more than 10.000), but at any
>> given query a maximum of say 20 are returned (through fl-field ) or used
>> (as part of filtering, faceting, sorting)
>> 
> 
> 
> Hi Britske, how do you manage 10.000 field  definitions? Sorry I didn't
> understand...
> 

I use dynamic fields. My 10k fields span all possible combinations of
variables, say, x, y, z. 
Then I can uniquely determine a column by specifying: _d_<x>_<y>_<z> for
example. 
while the field def is simply: <dynamicField name="_d_*" type="sint"
stored="true" indexed="true"/>
	

Funtick wrote:
> 
> Guys, I am constantly seeing the same problem, athough I have just a few
> small fields defined, lazyLoading is disabled, and memory is more than
> enough (25Gb for SOLR, 7Gb for OS, 3Gb index).
> 
> Britske, do you see the difference with faceted queries only?
> 

No, the difference is there all the time (faceting or not), but it gets vary
noticable when a lot of rows are returned. As commented earlier, it is
highly likely that this is due to random disk seek times. Because a lot of
fields are stored in my situation the harddisk has to cover many blocks to
fetch all requested documents.  


Funtick wrote:
> 
> Yonik, 
> 
> I am suspecting there is _bug_ with SOLR faceting so that faceted query
> time (qtime) is 10-20ms and elapsed time is huge; SOLR has filterCache
> where Key is 'filter'; SOLR does not have any <queryFacetResultCache>
> where Key is 'query' and Value is 'facets'...
> 
> Am I right?
> 
> -Fuad
> 
> 

-- 
View this message in context: http://www.nabble.com/big-discrepancy-between-elapsedtime-and-qtime-although-enableLazyFieldLoading%3D-true-tp18698590p18736439.html
Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message