lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Scaling data extractor with Solr
Date Tue, 04 Oct 2016 03:08:51 GMT
You can have as many clients indexing to Solr (either Cloud or
stand-alone) as you want, limited only by the load you put
on Solr. I.e. if your indexing throughput is so great that it makes
querying too slow then you have to scale back...

I know of setups with 100+ separate clients all indexing to Solr
at the same time.

Best,
Erick

On Mon, Oct 3, 2016 at 3:13 PM, Steven White <swhite4141@gmail.com> wrote:
> Hi everyone,
>
> I'm up to speed about Solr on how it can be setup to provide high
> availability (if one Solr server goes down, the backup one takes over).  My
> question is how do I make my custom crawler to play "nice" with Solr in
> this environment.
>
> Let us say I setup Solr with 3 servers so that if one fails the other one
> takes over.  Let us say I also setup my crawler with 3 servers so if one
> goes down the other takes over.  But how should my crawlers work?  Can each
> function unaware of each other and send the same data to Solr or must my
> crawlers synchronize with each other so only 1 is active sending data to
> Solr and the others are on stand-by mode?
>
> I like to hear from others how they solved this problem so I don't end up
> re-inventing it.
>
> Thanks.
>
> Steve

Mime
View raw message