lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] Updated: (LUCENE-2788) Make CharFilter reusable
Date Wed, 01 Dec 2010 20:45:12 GMT


Robert Muir updated LUCENE-2788:

    Attachment: LUCENE-2788.patch

here's a very quick patch (all tests pass):
* Changed CharFilter to extend FilterReader, and removed CharStream and CharReader.
* added reset(Reader) to CharFilter, so you can reset your entire charfilter chain with a
new reader.
* changed Solr to re-use the charfilter chain in its Analyzers.

it would be nice to add specific reuse tests to some of these charfilters, and also to see
if theres a way we can do this with any backwards compatibility... i didnt worry about that
when making the patch.

> Make CharFilter reusable
> ------------------------
>                 Key: LUCENE-2788
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Robert Muir
>            Priority: Minor
>         Attachments: LUCENE-2788.patch
> The CharFilter API lets you wrap a Reader, altering the contents before the Tokenizer
sees them.
> It also allows you to correct the offsets so this is transparent to highlighting.
> One problem is that the API isn't reusable, if you have a lot of short documents its
going to be efficient.
> Additionally there is some unnecessary wrapping in Tokenizer (see the CharReader.get
in the ctor, but *not* in reset(Reader)!!!)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message