lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ensdorf Ken <Ensd...@zoominfo.com>
Subject RE: Question about index sizes.
Date Tue, 23 Jun 2009 18:59:25 GMT
That's a great question.  And the answer is, of course, it depends.  Mostly on the size of
the documents you are indexing.  50 million rows from a database table with a handful of columns
is very different from 50 million web pages,  pdf documents, books, etc.

We currently have about 50 million documents split across 2 servers with reasonable performance
- sub-second response time in most cases.  The total size of the 2 indices is about 300G.
 I'd say most of the size is from stored fields, though we index just about everything.  This
is on 64-bit ubuntu boxes with 32G of memory.  We haven't pushed this into production yet,
but initial load-testing results look promising.

Hope this helps!

> -----Original Message-----
> From: Jim Adams [mailto:jasolruser@gmail.com]
> Sent: Tuesday, June 23, 2009 1:24 PM
> To: solr-user@lucene.apache.org
> Subject: Question about index sizes.
>
> Can anyone give me a rule of thumb for knowing when you need to go to
> multicore or shards?  How many records can be in an index before it
> breaks
> down?  Does it break down?  Is it 10 million? 20 million?  50 million?
>
> Thanks, Jim

Mime
View raw message