lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "DM Smith (JIRA)" <>
Subject [jira] Commented: (LUCENE-1350) SnowballFilter resets the payload
Date Wed, 06 Aug 2008 21:42:46 GMT


DM Smith commented on LUCENE-1350:

Non-reuse TokenStream API is not deprecated in the trunk. I guess you mean it will be deprecated
by LUCENE-1333.

My argument for actually deprecating it in LUCENE-1333 was that it was implicitly deprecated

So I wonder what's next with this issue. The complete LUCENE-1333 is dated for 2.4. So it
seems in place to fix filters behavior now, to preserve payload (and flags, thanks for pointing
this out), following the above (reuse) code pattern. Makes sense?

Yes. This issue is a bug fix and if there is a bug release, this should go with it.

I think that you should expand this issue to fix all the other TokenFilters that have the
same problem. And to also propagate the flags as well. As you found out a fix is trivial.
If you want, I'd be happy to work up a patch for it.

Regarding LUCENE-1333:
BTW, it can go before 2.4. I just set it to 2.4 since that was the next non-bug release.

It will be easy for me to adjust it after this goes in.

I'll change the JavaDoc to be clearer as to when clear() should be called, what is a producer
and what is a consumer, as part of it since that is part of it's scope.

> SnowballFilter resets the payload
> ---------------------------------
>                 Key: LUCENE-1350
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Analysis, contrib/*
>            Reporter: Doron Cohen
>            Assignee: Doron Cohen
>         Attachments: LUCENE-1350.patch
> Passing tokens with payloads through SnowballFilter results in tokens with no payloads.
> A workaround for this is to apply stemming first and only then run whatever logic creates
the payload, but this is not always convenient.
> Patch to follow that preserves the payload.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message