lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Isakson" <>
Subject RE: Should Token be immutable?
Date Tue, 07 Jan 2003 17:02:09 GMT
Hi All,

When you are creating new tokens to place in the same position, you could probably do this
by adding a constructor that accepts the position increment. This would allow you to keep
the object immutable.

OTOH, I think you still need the setPositionIncrement(int) for times when you want to count
dropped stop words and set the position increment on the next non-stop word token without
creating a new Token object just to set this value. Of course, in the current implementation,
we create a new Token if we want to modify the termText, so the same strategy could be used
by using the constructor mentioned above when the position increment needs to change.

The origin of this thread came about when I was responding to another user that was making
new token objects in their TokenFilter and I noticed the LowerCaseFilter just set the package
scoped member variable termText of the Token object.

It seems to me that Token could be made immutable by adding the positionIncrement to the constructor
and using that to create new tokens as needed.

It also seems reasonable to me that termText should be private and Token could be mutable
with setters for each property.

Sorry to waste cycles on this if folks think this is really a non-issue. Use of Token objects
wasn't consistent and it threw me off when I was learning the lucene code. I think that is
the real issue that was being raised.

Eric D. Isakson        SAS Institute Inc.
Application Developer  SAS Campus Drive
XML Technologies       Cary, NC 27513
(919) 531-3639

To unsubscribe, e-mail:   <>
For additional commands, e-mail: <>

View raw message