lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Britske <gbr...@gmail.com>
Subject Re: big discrepancy between elapsedtime and qtime although enableLazyFieldLoading= true
Date Mon, 28 Jul 2008 21:29:54 GMT

Thanks for clearing that up for me.
I'm going to investigate some more...



Yonik Seeley wrote:
> 
> On Mon, Jul 28, 2008 at 4:53 PM, Britske <gbrits@gmail.com> wrote:
>> Each query requests at most 20 stored fields. Why doesn't help
>> lazyfieldloading in this situation?
> 
> It's the disk seek that kills you... loading 1 byte or 1000 bytes per
> document would be about the same speed.
> 
>> Also, if I understand correctly, for optimal performance I need to have
>> at
>> least enough RAM to put the entire Index size in OS cache (thus RAM) +
>> the
>> amount of RAM that SOLR / Lucene consumes directly through the JVM?
> 
> The normal usage is to just retrieve the stored fields for the top 10
> (or a window of 10 or 20) documents.  Under this scenario, the
> slowdown from not having all of the stored fields cached is usually
> acceptable.  Faster disks (seek time) can also help.
> 
>> Luckily most of the normal queries return 10 documents each, which
>> results
>> in a discrepancy between total elapsed time and qTIme of about 15-30 ms.
>> Doesn't this seem strange, since to me it would seem logical that the
>> discrepancy would be at least 1/10th of fetching 100 documents.
> 
> Yes, in general 1/10th the cost is what one would expect on average.
> But some of the docs you are trying to retrieve *will* be in cache, so
> it's hard to control this test.
> You could try forcing the index out of memory by "cat"ing some other
> big files multiple times and then re-trying.... or do a reboot to be
> sure.
> 
> -Yonik
> 
> 

-- 
View this message in context: http://www.nabble.com/big-discrepancy-between-elapsedtime-and-qtime-although-enableLazyFieldLoading%3D-true-tp18698590p18699991.html
Sent from the Solr - User mailing list archive at Nabble.com.


Mime
View raw message