lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: Lucene indexing speed on NVMe drive
Date Thu, 30 Apr 2015 18:32:27 GMT

: Hi. I am studying Lucene performance and in particular how it benefits from faster I/O such
as SSD and NVMe.

: parameters as used in nightlyBench. (Hardware: Intel Xeon, 2.5GHz, 20 
: processor ,40 with hyperthreading, 64G Memory) and study indexing speed 
	...
: I get best performance (200GB/hour) with 20 indexing threads, increasing 
: number of threads to 40 hurts performance. Similarly increasing 
: maxConcurrentMerges above 3-5 doesn't seem to give me any benefit. I am 
: wondering what the bottleneck is, or anyone has insight on set of 

Maybe i'm missing something, but it sounds like you are CPU bound.  

Hyperthreading isn't going to help you if you are maxing out 20 (real) 
CPUS -- IIUC it only helps with some additional paralellization when 
processes are blocked by something else -- ie: IO bound.




-Hoss
http://www.lucidworks.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message