lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-8395) WordDelimiterGraphFilter can incorrectly add holes to a TokenStream
Date Thu, 12 Jul 2018 12:23:00 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-8395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541544#comment-16541544
] 

ASF subversion and git services commented on LUCENE-8395:
---------------------------------------------------------

Commit fd91ed4013c089bcf667372d55bec304dd37300f in lucene-solr's branch refs/heads/branch_7x
from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fd91ed4 ]

LUCENE-8395: WordDelimiterGraphFilter can incorrectly add holes


> WordDelimiterGraphFilter can incorrectly add holes to a TokenStream
> -------------------------------------------------------------------
>
>                 Key: LUCENE-8395
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8395
>             Project: Lucene - Core
>          Issue Type: New Feature
>            Reporter: Alan Woodward
>            Assignee: Alan Woodward
>            Priority: Major
>         Attachments: LUCENE-8395.patch
>
>
> If a token consists entirely of delimiter characters, then WordDelimiterGraphFilter
will remove the token and insert a hole into the TokenStream.  However, it does this even
if {{preserve_original}} is set, which results in an extra gap being added to the stream.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message