lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis_gospodne...@yahoo.com>
Subject Re: EmbeddedSolrServer and BinaryRequestWriter
Date Thu, 14 Jan 2010 23:10:29 GMT
Hi,

Running out of memory because of XML document when indexing documents sounds very weird/suspicious.
Are you running out of memory on the server side?
Are you indexing super large batches?
How big is your JVM heap?
How big is your ramBufferSizeMB?


Otis 
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch



----- Original Message ----
> From: Phil Hagelberg <phil@hagelb.org>
> To: solr-user@lucene.apache.org
> Sent: Thu, January 14, 2010 5:57:31 PM
> Subject: EmbeddedSolrServer and BinaryRequestWriter
> 
> 
> I'm trying to reduce memory usage when indexing, and I see that using
> the binary format may be a good way to do this. Unfortunately I can't
> see a way to do this using the EmbeddedSolrServer since only the
> CommonsHttpSolrServer has a setRequestWriter method. If I'm running out
> of memory constructing XML request documents, does that mean I just have
> to switch away from the EmbeddedSolrServer?
> 
> I understand I can stream requests if I'm just indexing files already on
> disk, but I'm constructing them on the fly, and I run out of memory
> constructing the XML document to submit to solr, not in actual indexing,
> so it seems writing the document to disk would run into the same problems.
> 
> thanks,
> Phil


Mime
View raw message