lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Facet counts and RankQuery
Date Tue, 21 Oct 2014 19:04:06 GMT
This is contrived I admit, but let's say you have
a query with 100 hits with a score distribution of
1 doc with a score of 100
98 docs with a score of 91
1 doc with a score of 1

Now I get 99 docs in my results set. Next I delete the
doc that scored 1 and my returned doc set
_for the exact same query_ will contain 1 doc.
How do you explain that to your user?

bq: We have other requirements regarding precision and
recall, especially when other sorts are specified.
So need to suppress docs based on thresholds.

I really don't understand this bit
"especially when other sorts are specified"

When you specify other sorts, you say in effect that
scoring is not as relevant. You're potentially making the
problem even worse by trying to mix other kinds of
sorting with some arbitrary cutoff. Let's take the
example of sorting by day into buckets (just day, not time).
How do you guarantee that a particular day has some kind of
representation? Could it happen that  in the last week
no doc happens to score above the 20th
percentile? What happens then, you show _no_ docs
that have a date of the last week? Or put some kind of
logic in the query like "well, if no doc scoring above 20%
happens to show up when respecting the 'other sort',
we'll just display some with lower scores?" If something like
that is a rule, how to explain inconsistent results to a
user? The user can quite logically say "When I search some
ORed terms I see doc X from yesterday, but when I search
some other ORed terms it doesn't show up even though I
see the some terms from the second query in the first result
set in doc X".

I'm sure your problem is more complex than the simple
example above, but you see the idea.

The other consideration is that thresholds tell you almost
nothing about the "goodness" of the score.

Consider query A with a max score of 100 and a min
score of 0.000001.

Now consider query  B with a max score of 100 and a min
score of 90.

Why would throwing away docs that scored < 92
for the second query be in any way comparable to
throwing away docs with a score < 20 in the first?

Even if you normalized them to scores between 0 and 1,
the argument holds.

That said, you know your problem space better than I
do, it's just that usually this kind of requirement is of
little practical value, often requiring far more effort
than the benefit _to the end user_ merits.

All _that_ said, nobody can speak cogently to your
"other requirements regarding precision and recall"
unless you tell us what they _are_, but that's
entirely up to you of course.

Best,
Erick

On Tue, Oct 21, 2014 at 1:42 PM, Parvesh Garg <parvesh@zettata.com> wrote:
> Hi Joel,
>
> Thanks for the pointer. Can you point me to any example implementation.
>
>
> Parvesh Garg,
> Founding Architect
>
> http://www.zettata.com
>
> On Tue, Oct 21, 2014 at 9:32 PM, Joel Bernstein <joelsolr@gmail.com> wrote:
>
>> The RankQuery cannot be used as filter. It is designed for custom
>> ordering/ranking of results only. If it's used as filter the facet counts
>> will not match up. If you need a filter collector then you need to use a
>> PostFilter.
>>
>> Joel Bernstein
>> Search Engineer at Heliosearch
>>
>> On Tue, Oct 21, 2014 at 10:50 AM, Erick Erickson <erickerickson@gmail.com>
>> wrote:
>>
>> > I _very strongly_ recommend that you do _not_ do this.
>> >
>> > First, the "problem" of having documents in the results
>> > list with, say, scores < 20% of the max takes care of itself;
>> > users stop paging pretty quickly. You're arbitrarily
>> > denying the users any chance of finding some documents
>> > that _do_ match their query. A user may know that a
>> > doc is in the corpus but be unable to find it. Very bad from
>> > a confidence-building standpoint.
>> >
>> > I've seen people put, say, 1-5 stars next to docs in the result
>> > to give the user some visual cue that they're getting into "less
>> > good" matches, but even that is of very limited value IMO. The
>> > stars represent quintiles, 5 stars for docs > 80% of max, 4
>> > stars between 60% and 80% etc.
>> >
>> > If you insist on this, then you'll need to run two passes
>> > across the data, the first will get the max score and the second
>> > will have a custom collector that somehow gets this number
>> > and rejects any docs below the threshold.
>> >
>> > Bet,
>> > Erick
>> >
>> > On Tue, Oct 21, 2014 at 3:09 AM, Parvesh Garg <parvesh@zettata.com>
>> wrote:
>> > > Hi All,
>> > >
>> > > We have written a RankQuery plugin with a custom TopDocsCollector to
>> > > suppress documents below a certain threshold w.r.t. to the maxScore for
>> > > that query. It works fine and is reflected well with numFound and start
>> > > parameters.
>> > >
>> > > Our problem lies with facet counts. Even though the docs numFound
>> gives a
>> > > very less number, the facet counts are still coming from unsuppressed
>> > query
>> > > results.
>> > >
>> > > E.g. in a test with a threshold of 20% , we reduced the totalDocs from
>> > > 46030 to 6080 but the top facet count on a field is still 20500
>> > >
>> > > The query parameter we are using looks like rq={!threshold value=0.2}
>> > >
>> > > Is there a way propagate the suppression of results to FacetsComponent
>> as
>> > > well? Can we send the same rq to FacetsComponent ?
>> > >
>> > >
>> > >
>> > > Regards,
>> > > Parvesh Garg,
>> > >
>> > > http://www.zettata.com
>> >
>>

Mime
View raw message