lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: maximum number of documents per shard?
Date Thu, 25 Jul 2013 12:07:02 GMT
I don't think there is any hard limit, but it will be more of a 
performance-based limit. Going beyond a couple dozen shards (lets say, 25) 
would take you into uncharted territory, where a sophisticated proof of 
concept implementation is essential. "Hundreds" or "thousands" of shards are 
likely to be problematic from a performance perspective for average users. 
In fact, I'd say that 8 shards may be a semi-practical limit, beyond which 
the design switches from "a walk in the park" to "heroic efforts". I mean, 
you should be able to do 25 shards, but in practice you will have to be much 
more alert, more careful with your hardware selection and network design, 
etc.

-- Jack Krupansky

-----Original Message----- 
From: Nicole Lacoste
Sent: Thursday, July 25, 2013 4:14 AM
To: solr-user@lucene.apache.org
Subject: Re: maximum number of documents per shard?

Is there a limit on the number of shards?

Niki


On 24 July 2013 01:14, Jack Krupansky <jack@basetechnology.com> wrote:

> 2.1 billion documents (including deleted documents) per Lucene index, but
> essentially per Solr shard as well.
>
> But don’t even think about going that high. In fact, don't plan on going
> above 100 million unless you do a proof of concept that validates that you
> get acceptable query and update performance . There is no hard limit
> besides that 2.1 billion Lucene limit, but... performance will vary.
>
> -- Jack Krupansky
>
> -----Original Message----- From: Ali, Saqib
> Sent: Tuesday, July 23, 2013 6:18 PM
> To: solr-user@lucene.apache.org
> Subject: maximum number of documents per shard?
>
> still 2.1 billion documents?
>



-- 
* <https://twitter.com/#!/niki_in_france>* 


Mime
View raw message