lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: MMapDirectory or FSDirectory
Date Thu, 05 Feb 2015 09:50:35 GMT
Hi,

If you use FSDirectory.open() it will automatically choose MMapDirectory on 64 bit systems.
Please note, virtual memory is != physical RAM. A 64 bit machine has *always* >1 Terabyte
of virtual address space available, this is unrelated to physical memory (a common misunderstanding
about mmap): http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

The speed difference depends on use case: In general MMapDirectory is much faster in multi-threaded
environments, because no concurrency problems. If you use SimpleFSDirectory this is the largest
bootleneck. NIOFSDirectory does not have concurrency problems, but it is still slower because
it does a lot of extra copying of data between kernel space and user space for buffering.
MMapDirectory is muuuuuuuch faster if you sort by docvalues fields, because it supports random
access without any buffering overhead.

So please: Use MMapDirectory where possible - this is completely unrelated to how much RAM
you have available!

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: sreedevi s [mailto:sreedevi.payikkad@gmail.com]
> Sent: Thursday, February 05, 2015 10:13 AM
> To: java-user@lucene.apache.org
> Subject: MMapDirectory or FSDirectory
> 
> Hi,
> I am doing some performance analysis with lucene. I have 1 million resources
> with 1000 attributes.
> According to how I index, I will have 1 million documents with 1000 fields.
> For me the total data was about 100 GB and while using FSDirectory to store
> my indices, index size was almost 6 GB.
> I have virtual memory available of almost 8 GB. Is it advised to use
> MMapDirectory for increased performance?
> Many blogs suggest it doesnt bring out much performance difference.
> 
> 
> Best Regards,
> Sreedevi S


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message