lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yonik Seeley" <ysee...@gmail.com>
Subject Re: WordDelimiterFilter looses position increments of tokens
Date Wed, 05 Jul 2006 14:17:04 GMT
On 7/5/06, Eric Jain <Eric.Jain@isb-sib.ch> wrote:
> Yonik Seeley wrote:
> > Your'e right, it does look possible that position info can be lost.
> > The fix probably isn't as simple as copying the position increment in
> > newTok()... the original increment should only be copied for the first
> > token generated.
>
> If something like
>
>    for (int i = 1, max = queue.size(); i < max; ++i)
>      queue.get(i).setPositionIncrement(1);
>
> were added just at the end of the next() method, before returning
> queue.get(0), that should fix the problem?

Not quite... all the position increments in the queue should be
correct (some 1 and some 0) except for potentially the first.

So fixing the first token at the end of next() and also at the other
exit point (line 276) is probably the easiest fix.

-Yonik

Mime
View raw message