lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <>
Subject Re: Interesting OutOfMemoryError on a 170M index
Date Wed, 13 Jan 2010 22:28:16 GMT
The time in autocommit is in milliseconds. You are committing every
second while indexing.  This then causes a build-up of sucessive index
readers that absorb each commit, which is probably the out-of-memory.

On Wed, Jan 13, 2010 at 10:36 AM, Minutello, Nick
<> wrote:
> Hi,
> I have a bit of an interesting OutOfMemoryError that I'm trying to
> figure out.
> My client & Solr server are running in the same JVM (for deployment
> simplicity). FWIW, I'm using Jetty to host Solr. I'm using the supplied
> code for the http-based client interface. Solr 1.3.0.
> My app is adding about 20,000 documents per minute to the index - one at
> a time (it is listening to an event stream and for every event, it adds
> a new document to the index).
> The size of the documents, however, is tiny - the total index growth is
> only about 170M (after about 1 hr and the OutOfMemoryError)
> At this point, there is zero querying happening - just updates to the
> index (only adding documents, no updates or deletes)
> After about an hour or so, my JVM runs out of heap space - and if I look
> at the memory utilisation over time, it looks like a classic memory
> leak. It slowly ramps up until we end up with constant FULL GC's and
> eventual OOME. Max heap space is 512M.
> In Solr, I'm using autocommit (to buffer the updates)
>        <autoCommit>
>          <maxDocs>10000</maxDocs>
>          <maxTime>1000</maxTime>
>        </autoCommit>
> (Aside: Now, I'm not sure if I am meant to call commit or not on the
> client SolrServer class if I am using autocommit - but as it turns out,
> I get OOME whether I do that or not)
> Any suggestions/advice of quick things to check before I dust off the
> profiler?
> Thanks in advance.
> Cheers,
> Nick
> ===============================================================================
>  Please access the attached hyperlink for an important electronic communications disclaimer:
>  ===============================================================================

Lance Norskog

View raw message