lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] [Commented] (LUCENE-4396) BooleanScorer should sometimes be used for MUST clauses
Date Wed, 30 Apr 2014 15:26:15 GMT


Michael McCandless commented on LUCENE-4396:

bq. I haven't merged my branch to the newest trunk version, because my network account at
school for April has been run out and I couldn't pull the code from github untill 1 May. Sorry
for that.

Good grief, that's awful you have to "budget" your allowed network
access month by month ... don't worry about it, it's easy to apply
the patch.  Alternatively, just add a pointer here to your github fork
and we can clone from there to review?

bq. I'm very sorry for the code style. That's my fault. Very sorry for that.

No need to apologize; it's just code style ... everyone has their own,
but we have a standard one in Lucene so we don't spend all our time
fighting over whose style is best :)

If you mean the .advance method of BooleanNovelScorer itself, I think it would be confusing,

because BooleanNovelScorer now is used when there's at least one MUST clause, 
no matter whether it acts as a top scorer or not. Therefore, .advance() of BooleanNovelScorer

must be called when BooleanNovelScorer acts as a non-top scorer.

Yeah I meant BNS.advance, when it's a top scorer.  Ie, can BNS beat
BS2 in this case.  It seems like you could test this case now since as
a topScorer nobody would call the unfinished BNS.advance method.  This
way, if BNS can beat BS2 in this case we know it's worth pursuing.

bq. I think this issue should be dealed with together.

+1, but only if time this summer permits (i.e. top priority is this
issue, allowing BS to accept MUST clauses).

> BooleanScorer should sometimes be used for MUST clauses
> -------------------------------------------------------
>                 Key: LUCENE-4396
>                 URL:
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>         Attachments: LUCENE-4396.patch, LUCENE-4396.patch
> Today we only use BooleanScorer if the query consists of SHOULD and MUST_NOT.
> If there is one or more MUST clauses we always use BooleanScorer2.
> But I suspect that unless the MUST clauses have very low hit count compared to the other
clauses, that BooleanScorer would perform better than BooleanScorer2.  BooleanScorer still
has some vestiges from when it used to handle MUST so it shouldn't be hard to bring back this
capability ... I think the challenging part might be the heuristics on when to use which (likely
we would have to use firstDocID as proxy for total hit count).
> Likely we should also have BooleanScorer sometimes use .advance() on the subs in this
case, eg if suddenly the MUST clause skips 1000000 docs then you want to .advance() all the
SHOULD clauses.
> I won't have near term time to work on this so feel free to take it if you are inspired!

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message