lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Hugo <m...@piragua.com>
Subject ConcurrentUpdateSolrServer flush on size of documents rather than queue size
Date Fri, 01 Mar 2013 16:30:55 GMT
Does anyone know if a version of ConcurrentUpdateSolrServer exists that
would use the size in memory of the queue to decide when to send documents
to the solr server?

For example, if I set up a ConcurrentUpdateSolrServer with 4 threads and a
batch size of 200 that works if my documents are small.  But if I am
building up documents that have a lot of text, I have run into an
OutOfMemory exception in my process that builds the docs.  The document
sizes are variable.

What I'd like to be able to do is submit documents to the solr sever when
the size of the queue reaches (or is greater than) 200MB or something like
that, so rather than specifying the number of document to put in the queue,
I'd specify the size in MB to build up before submitting.

Does something like this exist already?

Thanks,

Mike

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message