lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Injecting synonymns into Solr
Date Mon, 04 May 2015 13:22:32 GMT
On 5/4/2015 12:07 AM, Zheng Lin Edwin Yeo wrote:
> Would like to check, will this method of splitting the synonyms into
> multiple files use up a lot of memory?
> 
> I'm trying it with about 10 files and that collection is not able to be
> loaded due to insufficient memory.
> 
> Although currently my machine only have 4GB of memory, but I only have
> 500,000 records indexed, so not sure if there's a significant impact in the
> future (even with larger memory) when my index grows and other things like
> faceting, highlighting, and carrot tools are implemented.

For Solr, depending on exactly how you use it, the number of docs, and
the nature of those docs, a 4GB machine will usually be considered quite
small.  Solr requires a big chunk of RAM for its heap, but usually
requires an even larger chunk of RAM for the OS disk cache.

My Solr machines have 64GB (as much as the servers can hold) and I wish
they had two or four times as much, so I could get better performance.
My larger indexes (155 million docs, 103 million docs, and 18 million
docs, not using SolrCloud) are NOT considered very large by this
community -- we have users wrangling billions of docs with SolrCloud,
using hundreds of servers.

On this Wiki page, I have tried to outline how various aspects of memory
can affect Solr performance:

http://wiki.apache.org/solr/SolrPerformanceProblems

Thanks,
Shawn


Mime
View raw message