lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thorsten Scherler <thorsten.scherler....@juntadeandalucia.es>
Subject Re: Indexing very large files.
Date Thu, 06 Sep 2007 07:22:20 GMT
On Thu, 2007-09-06 at 08:55 +0200, Brian Carmalt wrote:
> Hello again,
> 
> I run Solr on Tomcat under windows and use the tomcat monitor to start 
> the service. I have set the minimum heap
> size to be 512MB and then maximum to be 1024mb. The system has 2 Gigs of 
> ram. The error that I get after sending
> approximately 300 MB is:
> 
> java.lang.OutOfMemoryError: Java heap space
>     at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2947)
>     at org.xmlpull.mxp1.MXParser.more(MXParser.java:3026)
>     at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1384)
>     at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
>     at org.xmlpull.mxp1.MXParser.nextText(MXParser.java:1058)
>     at 
> org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequestHandler.java:332)
>     at 
> org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestHandler.java:162)
>     at 
> org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:84)
>     at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77)
>     at org.apache.solr.core.SolrCore.execute(SolrCore.java:658)
>     at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191)
>     at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159)
>     at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
>     at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
>     at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
>     at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
>     at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>     at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
>     at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>     at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:261)
>     at 
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
>     at 
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:581)
>     at 
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
>     at java.lang.Thread.run(Thread.java:619)
> 
> After sleeping on the problem I see that it does not directly stem from 
> Solr, but from the
> module  org.xmlpull.mxp1.MXParser. Hmmm. I'm open to sugestions and ideas.

Which version do you use of solr?

http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/handler/XmlUpdateRequestHandler.java?view=markup

The trunk version of the XmlUpdateRequestHandler is now based on StAX.
You may want to try whether that is working better.

Please try and report back.

salu2
-- 
Thorsten Scherler                                 thorsten.at.apache.org
Open Source Java                      consulting, training and solutions


Mime
View raw message