lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghuveer Kancherla <raghuveer.kanche...@aplopio.com>
Subject Re: Retrieving large num of docs
Date Thu, 03 Dec 2009 13:43:16 GMT
Hi Hoss,

I was experimenting with various queries to solve this problem and in one
such test I remember that requesting only the ID did not change the
retrieval time. To be sure, I tested it again using the curl command today
and it confirms my previous observation.

Also, enableLazyFieldLoading setting is set to true in my solrconfig.

Another general observation (off topic) is that having a moderately large
multi valued text field (~200 entries) in the index seems to slow down the
search significantly. I removed the 2 multi valued text fields from my index
and my search got ~10 time faster. :)

- Raghu


On Thu, Dec 3, 2009 at 2:14 AM, Chris Hostetter <hossman_lucene@fucit.org>wrote:

>
> : I think I solved the problem of retrieving 300 docs per request for now.
> The
> : problem was that I was storing 2 moderately large multivalued text fields
> : though I was not retrieving them during search time.  I reindexed all my
> : data without storing these fields. Now the response time (time for Solr
> to
> : return the http response) is very close to the QTime Solr is showing in
> the
>
> Hmmm....
>
> two comments:
>
> 1) the example URL from your previous mail...
>
> : >
> http://localhost:1212/solr/select/?rows=300&q=%28ResumeAllText%3A%28%28%28%22java+j2ee%22+%28java+j2ee%29%29%29%5E4%29%5E1.0%29&start=0&wt=python
>
> ...doesn't match your earlier statemnet that you are only returning hte id
> field (there is no "fl" param in that URL) ... are you certain you werent'
> returning those large stored fields in teh response?
>
> 2) assuming you were actually using an fl param to limit the fields, make
> sure you have this setting in your solrconfig.xml...
>
>    <enableLazyFieldLoading>true</enableLazyFieldLoading>
>
> ..that should make it pretty fast to return only a few fields of each
> document, even if you do have some jumpto stored fields that aren't being
> returned.
>
>
>
> -Hoss
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message