lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <yo...@heliosearch.com>
Subject Re: Details on why ConccurentUpdateSolrServer is reccommended for maximum index performance
Date Thu, 11 Dec 2014 16:40:02 GMT
On Wed, Dec 10, 2014 at 6:09 PM, Erick Erickson <erickerickson@gmail.com> wrote:
> So CUSS will do something like this:
> 1> assemble a packet for Solr
> 2> pass off the actual transmission
>      to Solr to a thread and immediately
>      go back to <1>.
>
> Basically, CUSS is doing async processing.

The more important part about what it's doing is the *streaming*.
CUSS is like batching documents without waiting for all of the
documents in the batch.
When you add a document, it immediately writes it to a stream where
solr can read it off and index it.  When you add a second document,
it's immediately written to the same stream (or at least one of the
open streams), as part of the same udpate request.  No separate HTTP
request, No separate update request.

The number of threads parameter for CUSS actually maps to the number
of open connections to Solr (and hence the number of concurrently
streaming update requests).

So to Solr (server side), it looks like a single update request
(assuming 1 thread) with a batch of multiple documents... but it was
never actually "batched" on the client side.

-Yonik

Mime
View raw message