lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars Clausen ...@statsbiblioteket.dk>
Subject Re: OutOfMemoryError on small search in large, simple index
Date Mon, 07 Jan 2008 10:00:54 GMT

On Tue, 2008-01-01 at 23:38 -0800, Chris Hostetter wrote:
> : On Wed, 2007-12-12 at 11:37 +0100, Lars Clausen wrote:
> 
> : Seems there's a reason we still use all this memory:
> : SegmentReader.fakeNorms() creates the full-size array for us anyway, so
> : the memory usage cannot be avoided as long as somebody asks for the
> : norms array at any point.  The solution is starting to look like a patch
> 
> Minor (late) nit:  fakeNorms will reuse the same array for every field 
> that doesn't have norms -- so while omiting norms from only one field may 
> not save you any RAM, omiting norms for 10 fields will save you the RAM 
> used by 9 of those fields.  (correct?)

Doesn't appear to be the case in our test.  We had two fields with
norms, omitting saved only about 4MB for 50 million entries.  I'm going
to try the patch that's attached to a bug report somewhere, which fakes
the norms without making an array for it.

-Lars



---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message