lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Rodenburg" <jeff.rodenb...@gmail.com>
Subject Re: Troubleshooting java heap out-of-memory
Date Mon, 02 Apr 2007 21:14:38 GMT
On 4/2/07, Yonik Seeley <yonik@apache.org> wrote:
>
> On 4/1/07, Jeff Rodenburg <jeff.rodenburg@gmail.com> wrote:
> > Our scenario: 150MB index, 140000 documents, read/write servers in place
> > using standard replication.  Running Tomcat 5.5.17 on Redhat Enterprise
> > Linux 4.  Java configured to start with -Xmx1024m.  We encounter java
> heap
> > out-of-memory issues on the read server at staggered times, but usually
> once
> > every 48 hours.
>
> Could you do a grep through your server logs for "WARNING", to
> eliminate the possibility of multiple overlapping searchers causing
> the OOM issue?


We're not seeing warnings for overlapping searchers prior to the oom
events.  Only "SEVERE" -- java.lang.OutOfMemoryError: Java heap space.

Are you doing incremental updates?  If so, try lowering your
> mergeFactor for the index, or optimize more frequently.  As an index
> is incrementally updated, old docs are marked as deleted and new docs
> are added.  This leaves "holes" in the document id space which can
> increase memory usage.  Both BitSet filters and FieldCache entry sizes
> are proportionally related to maxDoc (the maximum internal docid in
> the index).
>
> You can see maxDoc from the statistics page... there might be a
> correlation.


We are doing incremental updates, and we optimize quite a bit.  mergeFactor
presently set to 10.
maxDoc count = 144156
numDocs count = 144145

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message