lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Male (JIRA)" <>
Subject [jira] [Updated] (LUCENE-3455) All Analysis Consumers should use reusableTokenStream
Date Mon, 26 Sep 2011 05:28:26 GMT


Chris Male updated LUCENE-3455:

    Attachment: LUCENE-3455.patch

Patch which converts the last consumers over to using reusableTokenStream().

Couple of changes to the classic QP included in this.  Rather than silently returning null
on an IOException, a ParseException is now thrown.  Equally, where before the IOException
was simply ignored, an Exception is thrown.

Going to commit this shortly and then change reusableTokenStream() to tokenStream().

> All Analysis Consumers should use reusableTokenStream
> -----------------------------------------------------
>                 Key: LUCENE-3455
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Sub-task
>          Components: modules/analysis
>            Reporter: Chris Male
>            Assignee: Chris Male
>         Attachments: LUCENE-3455-test-consumers.patch, LUCENE-3455-test-consumers.patch,
> With Analyzer now using TokenStreamComponents, theres no reason for Analysis consumers
to use tokenStream() (it just gives bad performance).  Consequently all consumers will be
moved over to using reusableTokenStream().  The only challenge here is that reusableTokenStream
throws an IOException which many consumers are not rigged to deal with.
> Once all consumers have been moved, we can rename reusableTokenStream() back to tokenStream().

This message is automatically generated by JIRA.
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message