lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Miller <markrmil...@gmail.com>
Subject Re: Too many open files
Date Fri, 23 Oct 2009 19:02:50 GMT
I wouldn't use a RAM buffer of a gig - 32-100 is generally a good number.

Fuad Efendi wrote:
> I was partially wrong; this is what Mike McCandless  (Lucene-in-Action, 2nd
> edition) explained at Manning forum:
>
> mergeFactor of 1000 means you will have up to 1000 segments at each level.
> A level 0 segment means it was flushed directly by IndexWriter.
> After you have 1000 such segments, they are merged into a single level 1
> segment.
> Once you have 1000 level 1 segments, they are merged into a single level 2
> segment, etc.
> So, depending on how many docs you add to your index, you'll could have
> 1000s of segments w/ mergeFactor=1000.
>
> http://www.manning-sandbox.com/thread.jspa?threadID=33784&tstart=0
>
>
> So, in case of mergeFactor=100 you may have (theoretically) 1000 segments,
> 10-20 files each (depending on schema)...
>
>
> mergeFactor=10 is default setting... ramBufferSizeMB=1024 means that you
> need at least double Java heap, but you have -Xmx1024m...
>
>
> -Fuad
>
>
>   
>> I am getting too many open files error.
>>
>> Usually I test on a server that has 4GB RAM and assigned 1GB for
>> tomcat(set JAVA_OPTS=-Xms256m -Xmx1024m), ulimit -n is 256 for this
>> server and has following setting for SolrConfig.xml
>>
>>
>>
>>     <useCompoundFile>true</useCompoundFile>
>>
>>     <ramBufferSizeMB>1024</ramBufferSizeMB>
>>
>>     <mergeFactor>100</mergeFactor>
>>
>>     <maxMergeDocs>2147483647</maxMergeDocs>
>>
>>     <maxFieldLength>10000</maxFieldLength>
>>
>>     
>
>
>   


-- 
- Mark

http://www.lucidimagination.com




Mime
View raw message