lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lukas Vlcek (JIRA)" <j...@apache.org>
Subject [jira] [Created] (LUCENE-5484) Distinct control of recursion levels for prefix and suffix in Hunspell.
Date Sun, 02 Mar 2014 10:44:19 GMT
Lukas Vlcek created LUCENE-5484:
-----------------------------------

             Summary: Distinct control of recursion levels for prefix and suffix in Hunspell.
                 Key: LUCENE-5484
                 URL: https://issues.apache.org/jira/browse/LUCENE-5484
             Project: Lucene - Core
          Issue Type: Improvement
          Components: modules/analysis
            Reporter: Lukas Vlcek
            Priority: Minor


Currently, there is an option to set recursionCap value to control depth of recursion in Hunspell
token filter. This recursion enables to apply allowed affix rule to input token and pass output
token(s) as an input tokens recursively.

However, the recursionCap does not allow to distinguish between how many prefix and suffix
rules were applied. It just counts for total. For example if recursionCap is set to 1 it actually
includes all of the following options:
- 2 prefix rules, 0 suffix rules
- 1prefix rule, 1 suffix rule
- 0 prefix rules, 2 suffix rules

In some cases it is required to be able to distinguish between prefix rule and suffix rule
and have finer control over how many times is each applied. Requested feature should allow
setting recursion level separately for prefix and suffix rules.

Specific example is the Czech dictionary, where it gives best results if suffix rules are
applied only once. Hence recursionCap = 0. But if for input token a prefix rule is applied
it does not allow to apply suffix rule and produces a token that is not in root form. And
setting recursionCap = 1 produces too many irrelevant tokens that it makes Hunspell token
filter unuseful. Good solution to this problem would be tell Hunspell token filter to apply
up to 1 prefix rule and up to 1 suffix rule only (meaning never allow to apply 0 prefix rules
and 2 suffix rules).

Generally, this is probably dependant a lot on how particular dictionary and affix rules are
constructed so it might not be considered a generalization but rather and expert feature.

(There was some relevant discussion going on in LUCENE-5468)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message