lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Bharadwaj <mbharad...@gmail.com>
Subject Re: Advise on an architecture with lot of cores
Date Tue, 07 Oct 2014 14:26:48 GMT
Yes, we have plan to eventually have to shard the clusters - that will go
hand in hand with how rest of the system gets partitioned as well (swim
lanes). The other considerations for these lanes will be geo location etc
(in a AWS context, zones in east coast will be used for swim lanes that
cater to customer in that region.

p.s we are not yet in AWS, but it is part of medium term strategy.

On Tue, Oct 7, 2014 at 9:12 AM, Jack Krupansky <jack@basetechnology.com>
wrote:

> You'll have to do a proof of concept test to determine how many
> collections Solr/SolrCloud can handle.
>
> With a very large number of customers you may have to do sharding of the
> clusters themselves - limit each cluster to however many
> customers/colllections work well (100? 250?) and then have separate
> clusters for larger groups of customers, maybe with a smaller cluster with
> a collection that maps the customer ID to a Solr cluster, and then the
> application layer can direct requests to the Solr cluster that owns that
> customer.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Manoj Bharadwaj
> Sent: Tuesday, October 7, 2014 8:27 AM
> To: solr-user@lucene.apache.org
> Subject: Advise on an architecture with lot of cores
>
>
> Hi folks,
>
> My team inherited a SOLR setup with an architecture that has a core for
> every customer. We have a few different types of cores, say "A", "B", C",
> and for each one of this there is a core per customer - namely "A1",
> "A2"..., "B1", "B2"... Overall we have over 600 cores. We don't know the
> history behind the current design - the exact reasons why it was done the
> way it was done - one probable consideration was to ensure a customer data
> separate from other.
>
> We want to go to a single core per type architecture, and move on to  SOLR
> cloud as well in near future to achieve sharding via the features cloud
> provides.
>
> Further aspects such as monitoring become easier as well. We will need to
> watch and tune the caches for the different pattern of hits that we see.
>
> Is there anything else to evaluate before we move to a single core per type
> setup?
>
> We are using 4.4.0 currently and will be moving to latest 4.10.1 as a part
> of the redesign as well.
>
> Regards
> Manoj
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message