lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: ConcurrentUpdateSolrServer flush on size of documents rather than queue size
Date Sat, 02 Mar 2013 04:05:42 GMT
H Mike,

Doesn't exist as far as I know, but would be a nice contribution.

Otis
--
Solr & ElasticSearch Support
http://sematext.com/





On Fri, Mar 1, 2013 at 11:30 AM, Mike Hugo <mike@piragua.com> wrote:

> Does anyone know if a version of ConcurrentUpdateSolrServer exists that
> would use the size in memory of the queue to decide when to send documents
> to the solr server?
>
> For example, if I set up a ConcurrentUpdateSolrServer with 4 threads and a
> batch size of 200 that works if my documents are small.  But if I am
> building up documents that have a lot of text, I have run into an
> OutOfMemory exception in my process that builds the docs.  The document
> sizes are variable.
>
> What I'd like to be able to do is submit documents to the solr sever when
> the size of the queue reaches (or is greater than) 200MB or something like
> that, so rather than specifying the number of document to put in the queue,
> I'd specify the size in MB to build up before submitting.
>
> Does something like this exist already?
>
> Thanks,
>
> Mike
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message