lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Petersen" <rober...@buy.com>
Subject OOM on uninvert field request
Date Tue, 29 Jun 2010 23:32:27 GMT
Hello I am trying to find the right max and min settings for Java 1.6 on 20GB index with 8
million docs, running 1.6_018 JVM with solr 1.4, and am currently have java set to an even
4GB (export JAVA_OPTS="-Xmx4096m -Xms4096m") for both min and max which is doing pretty well
but occasionally still getting the below OOM errors.  We're running on dual quad core xeons
with 16GB memory installed.  I've been getting the below OOM exceptions still though.  

Is the memsize mentioned in the INFO for the uninvert in bytes?  Ie is memSize=29604020 mean
29MB?  We have a few hundred of these fields and they contain ints used as IDs, and so I guess
could they eat all the memory to uninvert them all after we apply load and enough queries
are performed.  Does the field type matter, would int be better than string if these are lookup
ids sparsely populated across the index?  BTW these are used for faceting and filtering only.

		<dynamicField name="*_contentAttributeToken"  type="string"  indexed="true" multiValued="true"
  stored="true" required="false"/>

Jun 29, 2010 3:54:50 PM org.apache.solr.request.UnInvertedField uninvert
INFO: UnInverted multi-valued field {field=768_contentAttributeToken,memSize=29604014,tindexSize=50,time=1841,phase1=1824,nTerms=1,bigTerms=0,termInstances=18,uses=0}
Jun 29, 2010 3:54:52 PM org.apache.solr.request.UnInvertedField uninvert
INFO: UnInverted multi-valued field {field=749_contentAttributeToken,memSize=29604020,tindexSize=56,time=1847,phase1=1829,nTerms=143,bigTerms=0,termInstances=951,uses=0}
Jun 29, 2010 3:54:59 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.OutOfMemoryError: Java heap space
	at org.apache.solr.request.UnInvertedField.uninvert(UnInvertedField.java:191)
	at org.apache.solr.request.UnInvertedField.<init>(UnInvertedField.java:178)
	at org.apache.solr.request.UnInvertedField.getUnInvertedField(UnInvertedField.java:839)
	at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:250)
	at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:283)
	at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:166)
Mime
View raw message