lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Høydahl (JIRA) <>
Subject [jira] [Commented] (LUCENE-3130) Use BoostAttribute in in TokenFilters to denote Terms that QueryParser should give lower boosts
Date Sun, 26 Jun 2011 09:46:47 GMT


Jan Høydahl commented on LUCENE-3130:

The feature is absolutely needed. Probably it's enough to be able to specify a global term
boost factor per query for all synonyms, so Robert's method would work for me.

Another usecase is Phonetic variants. Currently I use a separate field for phonetic normalization
and include it with a lower weight in DisMax. If phonetic variant instead was stored alongside
the original with posIncr=0 and tokenType=phonetic, I could instead specify a deboost factor
for phonetic terms and even highlighting would work ootb!

Yet another is lower/upper case search. If the LowerCaseFilter would keep the original token
and add a lowercased token on same posIncr with tokenType=lowercase, we could support case
insensitive match with preference for correct case.

If user needs different boost for different fields, perhaps the TokenType name could be configurable
on each filter.

> Use BoostAttribute in in TokenFilters to denote Terms that QueryParser should give lower
> -----------------------------------------------------------------------------------------------
>                 Key: LUCENE-3130
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Hoss Man
> A recent thread asked if there was anyway to use QueryTime synonyms such that matches
on the original term specified by the user would score higher then matches on the synonym.
 It occurred to me later that a float Attribute could be set by the SynonymFilter in such
situations, and QueryParser could use that float as a boost in the resulting Query.  IThis
would be fairly straightforward for the simple "synonyms => BooleamQuery" case, but we'd
have to decide how to handle the case of synonyms with multiple terms that produce MTPQ, possibly
just punt for now)
> Likewise, there may be other TokenFilters that "inject" artificial tokens at query time
where it also might make sense to have a reduced "boost" factor...
> * SynonymFilter
> * CommonGramsFilter
> * WordDelimiterFilter
> * etc...
> In all of these cases, the amount of the "boost" could me configured, and for back compact
could default to "1.0" (or null to not set a boost at all)
> Furthermore: if we add a new BoostAttrToPayloadAttrFilter that just copied the boost
attribute into the payload attribute, these same filters could give "penalizing" payloads
to terms when used at index time) could give "penalizing" payloads to terms.

This message is automatically generated by JIRA.
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message