lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benson Margulies <ben...@basistech.com>
Subject LookaheadTokenFilter
Date Fri, 06 Sep 2013 00:44:46 GMT
I'm trying to work through the logic of reading ahead until I've seen
marker for the end of a sentence, then applying some analysis to all of the
tokens of the sentence, and then changing some attributes of each token to
reflect the results.

The queue of tokens for a position is just a State, so there isn't an API
there to set any values.

So do I need to subclass Position for myself, store the additional
information in there, and set the attributes as each token comes by on the
output side?

I would be grateful for a bit more explanation of afterPosition versus
incrementToken; some of the mock classes call peek from afterPosition, and
I expected to see peek called in incrementToken based on the javadoc.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message