lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Ferenczi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-8772) [nori] A word that is registered in advance, but the words are not separated and recognized as 'UNKNOWN'
Date Fri, 19 Apr 2019 09:43:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16821810#comment-16821810
] 

Jim Ferenczi commented on LUCENE-8772:
--------------------------------------

That's expected since the unknown word heuristic is to group characters of the same class
together. In this case `갊수학` is considered as a single word and `갊` is unknown so
we jump to the end of the unknown word to find new entries. You can add `갊` in the user
dict or a special rule `갊수학 갊 수학` that will decompose the terms. We could also
change the heuristic to add unknown word of length 1 in order to be able to detect user words
inside unknown blocks but I wonder if the cost to do that is not prohibitive.

> [nori]  A word that is registered in advance, but the words are not separated and recognized
as 'UNKNOWN'
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-8772
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8772
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/analysis
>    Affects Versions: 7.5, 7.6, 7.7, 7.7.1, 8.0
>            Reporter: YOO JEONGIN
>            Priority: Major
>         Attachments: image-2019-04-19-11-32-56-310.png
>
>
> hello,
> In case of 'nori', if there is no word starting from the left, 'UNKNOWN' is analyzed
even if there is a word already registered in the middle.
>  So here is the question.
>  Does nori analyze only on the left side and do not analyze from the right side?
>  Could this be solved?
>  
> ex)
> input => 갊수학
> Condition
> dictionary registered : 수학
>  dictionary Unregistered : 갊
> result => 갊수학
> !image-2019-04-19-11-32-56-310.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message