lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oliver Flege <>
Subject FieldCacheImpl caveat
Date Mon, 07 Apr 2008 10:28:47 GMT

An undocumented feature of FieldCacheImpl led to an OutOfMemoryError 
in our application and to know about that might be of general interest:

We are using lucene 2.3.1 and our index (3m documents)
is updated 3 times per day. On every update, we create a new 
instance of a Searcher class, which is defined like this: 

class Searcher {
   private final IndexReader indexReader;
   private final int[] ids;

   Searcher(IndexReader indexReader) {
         this.indexReader = indexReader;

         this.ids = FieldCache.DEFAULT.getInts(this.indexReader, "ids",
                new FieldCache.IntParser() {
                    public int parseInt(String s) {
                        return Integer.parseInt(s, Character.MAX_RADIX);

The ids[] field is used to improve search speed and had been added recently.
At first, everything looked fine, but after a couple of days
our application crashed with an OutOfMemoryError.

The FieldCacheImpl (implementation behind FieldCache.DEFAULT) stores entries
in a WeakHashMap and uses IndexReader objects as keys, so when a particular 
IndexReader is only weakly reachable its values can be removed from the

Since we only have a single reference to the most recent Searcher instance
in a 
Controller class, it seemed obvious that stale values in FieldCache's
could be gc'ed as soon as a new Searcher object replaced the old one.

BUT FieldCacheImpl stores a reference to the IntParser (or any other parser)
in the cache. Since we used an anonymous inner class, it contained a
to the enclosing Searcher instance which in turn referenced the IndexReader,
that was never only weakly reachable and never got removed from the
After replacing the anonymous inner IntParser with a static inner class, 
the FieldCache worked as expected.

It would be a good idea to document the fact that the FieldCacheImpl stores
references to Parsers.


View this message in context:
Sent from the Lucene - Java Users mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message