lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From baris.ka...@oracle.com
Subject Re: MMapDirectory vs In Memory Lucene Index (i.e., ByteBuffersDirectory)
Date Tue, 23 Feb 2021 21:07:21 GMT
Thanks, but each different query i see some slowdown (not much though) 
with MMapDirectory and FSDirectory, though.

It is a little bit more with FSdirectory. So, MMapDirectory is slightly 
better in that, too: ie, cold start.


What i want to achieve: Problem statement:

base case is disk based Lucene index with FSDirectory

speedup case was supposed to be in memory Lucene index with MMapDirectory


Uwe mentioned tmpfs will help. i will try that next.

Thanks


On 2/23/21 3:54 PM, Robert Muir wrote:
> speedup over what? You are probably already using MMapDirectory (it is 
> the default). So I don't know what you are trying to achieve, but 
> giving lots of memory to your java process is not going to help.
>
> If you just want to prevent the first few queries to a fresh cold 
> machine instance from being slow, you can use the preload for that 
> before you make it available. You could also use 'cat' or 'dd'.
>
> On Tue, Feb 23, 2021 at 3:45 PM <baris.kazar@oracle.com 
> <mailto:baris.kazar@oracle.com>> wrote:
>
>     Thanks but then how will MMapDirectory help gain speedup?
>
>     i will try tmpfs and see what happens. i was expecting to get on
>     order of magnitude of speedup from already very fast on disk
>     Lucene indexes.
>
>     So i was expecting really really really fast response with
>     MMapDirectory.
>
>     Thanks
>
>
>     On 2/23/21 3:40 PM, Robert Muir wrote:
>>     Don't give gobs of memory to your java process, you will just
>>     make things slower. The kernel will cache your index files.
>>
>>     On Tue, Feb 23, 2021 at 1:45 PM <baris.kazar@oracle.com
>>     <mailto:baris.kazar@oracle.com>> wrote:
>>
>>         Ok, but how is this MMapDirectory used then?
>>
>>         Best regards
>>
>>
>>         On 2/23/21 7:03 AM, Robert Muir wrote:
>>         >
>>         >
>>         > On Tue, Feb 23, 2021 at 2:30 AM <baris.kazar@oracle.com
>>         <mailto:baris.kazar@oracle.com>
>>         > <mailto:baris.kazar@oracle.com
>>         <mailto:baris.kazar@oracle.com>>> wrote:
>>         >
>>         >     Hi,-
>>         >
>>         >       I tried MMapDirectory and i allocated as big as index
>>         size on my
>>         >     J2EE
>>         >     Container but
>>         >
>>         >
>>         > Don't allocate java heap memory for the index,
>>         MMapDirectory does not
>>         > use java heap memory!
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message