lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghuveer Kancherla <raghuveer.kanche...@aplopio.com>
Subject Re: payload queries running slow
Date Sun, 20 Dec 2009 08:41:39 GMT
Hi Grant,
My queries are about 5 times slower when using payloads as compared to
queries that dont use payloads on the same index. I have not done any
profiling yet, I am trying out lucid gaze now.
I do all the load testing after warming up.
Since my index is small ~1 GB, was wondering if a ramDirectory will help
instead of the default Directory implementation for the indexReader?

Thanks,
Raghu



On Thu, Dec 17, 2009 at 6:58 PM, Grant Ingersoll <gsingers@apache.org>wrote:

>
> On Dec 17, 2009, at 4:52 AM, Raghuveer Kancherla wrote:
>
> > Hi,
> > With help from the group here, I have been able to set up a search
> > application with payloads enabled. However, there is a noticeable
> increase
> > in query response times with payloads as compared to the same queries
> > without payloads. I am also seeing a lot more disk IO (I have a 7200 rpm
> > disk) and comparatively lesser cpu usage.
> >
> > I am guessing this is because of the use of payloadTermQuery and
> > payloadNearQuery  both of which extend SpanQuery formats. SpanQueries
> read
> > the positions index which will be much larger than the index accessed by
> a
> > simple TermQuery.
> >
> > Is there any way of making this system faster without having to
> distribute
> > the index. My index size is hardly 1GB (~200k documents and only one
> field
> > to search in). I am experiencing query times as high as 2 seconds
> (average).
> >
> > Any indications on the direction in which I can experiment will also be
> very
> > helpful.
> >
>
> Yeah, payloads are going to be slower, but how much slower are they for
> you? Are you warming up those queries?
>
> Also, have you done any profiling?
>
>
> > I looked at HathiTrust digital library articles. The methods indicated
> there
> > talk about avoiding reading the positions index (converting PhraseQueries
> to
> > TermQueries). That will not work in my case because, I still have to read
> > the positions index to get the payload information during scoring. Let me
> > know if my understanding is incorrect.
> >
> >
> > Thanks,
> > -Raghu
>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message