lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Bowyer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (LUCENE-3178) Native MMapDir
Date Fri, 07 Mar 2014 06:49:47 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13564880#comment-13564880
] 

Greg Bowyer edited comment on LUCENE-3178 at 3/7/14 6:48 AM:
-------------------------------------------------------------

So I was going to shut this down today, and just to make sure I ran the benchmark on the simplest
code possible

... and suddenly I got good results, this is idiopathic :S

{code}
Report after iter 19:
                    TaskQPS baseline      StdDevQPS mmap_tests      StdDev               
Pct diff
              OrHighHigh        1.68     (11.2%)        1.73     (10.3%)    3.0% ( -16% -
  27%)
                PKLookup      129.89      (5.8%)      135.03      (6.0%)    4.0% (  -7% -
  16%)
                HighTerm        8.09     (13.6%)        8.43     (12.8%)    4.2% ( -19% -
  35%)
               OrHighMed        4.46     (10.4%)        4.67      (9.5%)    4.7% ( -13% -
  27%)
               OrHighLow        4.82     (10.6%)        5.09     (10.3%)    5.6% ( -13% -
  29%)
            HighSpanNear        0.92      (8.1%)        0.97      (7.3%)    5.9% (  -8% -
  23%)
                  IntNRQ        2.51     (10.2%)        2.67      (9.9%)    6.6% ( -12% -
  29%)
              HighPhrase        0.30     (11.7%)        0.32     (12.8%)    6.7% ( -16% -
  35%)
               MedPhrase        2.93      (6.8%)        3.12      (8.2%)    6.7% (  -7% -
  23%)
             AndHighHigh        5.46      (6.6%)        5.86      (7.0%)    7.2% (  -5% -
  22%)
                 Respell       19.68      (5.9%)       21.15      (6.6%)    7.5% (  -4% -
  21%)
               LowPhrase        0.46      (9.5%)        0.50     (10.2%)    7.6% ( -11% -
  30%)
                 Prefix3        5.25      (8.2%)        5.66      (7.7%)    7.9% (  -7% -
  25%)
        HighSloppyPhrase        1.54      (8.0%)        1.67     (13.1%)    8.5% ( -11% -
  32%)
             MedSpanNear        5.25      (7.0%)        5.72      (8.2%)    9.0% (  -5% -
  25%)
                Wildcard       12.44      (5.7%)       13.59      (6.5%)    9.2% (  -2% -
  22%)
         MedSloppyPhrase        2.27      (7.2%)        2.49      (8.5%)    9.5% (  -5% -
  27%)
                 MedTerm       28.16     (10.3%)       30.89      (9.9%)    9.7% (  -9% -
  33%)
                  Fuzzy1       18.91      (6.0%)       20.82      (6.7%)   10.1% (  -2% -
  24%)
                  Fuzzy2       19.69      (6.6%)       21.68      (7.5%)   10.1% (  -3% -
  25%)
              AndHighMed        7.79      (7.5%)        8.58      (6.1%)   10.1% (  -3% -
  25%)
             LowSpanNear        1.45      (5.7%)        1.60      (9.3%)   10.5% (  -4% -
  27%)
         LowSloppyPhrase       22.84      (7.7%)       25.45      (9.7%)   11.4% (  -5% -
  31%)
                 LowTerm       46.46      (6.8%)       52.90      (7.6%)   13.9% (   0% -
  30%)
              AndHighLow       35.92      (5.3%)       42.38      (7.1%)   18.0% (   5% -
  32%)
{code}


was (Author: gbowyer@fastmail.co.uk):
So I was going to shut this down today, and just to make sure I ran the benchmark on the simplest
code possible

... and suddenly I got good results, this is idiopathic :S

https://gist.github.com/0f017853861d050c0b66
{code}
Report after iter 19:
                    TaskQPS baseline      StdDevQPS mmap_tests      StdDev               
Pct diff
              OrHighHigh        1.68     (11.2%)        1.73     (10.3%)    3.0% ( -16% -
  27%)
                PKLookup      129.89      (5.8%)      135.03      (6.0%)    4.0% (  -7% -
  16%)
                HighTerm        8.09     (13.6%)        8.43     (12.8%)    4.2% ( -19% -
  35%)
               OrHighMed        4.46     (10.4%)        4.67      (9.5%)    4.7% ( -13% -
  27%)
               OrHighLow        4.82     (10.6%)        5.09     (10.3%)    5.6% ( -13% -
  29%)
            HighSpanNear        0.92      (8.1%)        0.97      (7.3%)    5.9% (  -8% -
  23%)
                  IntNRQ        2.51     (10.2%)        2.67      (9.9%)    6.6% ( -12% -
  29%)
              HighPhrase        0.30     (11.7%)        0.32     (12.8%)    6.7% ( -16% -
  35%)
               MedPhrase        2.93      (6.8%)        3.12      (8.2%)    6.7% (  -7% -
  23%)
             AndHighHigh        5.46      (6.6%)        5.86      (7.0%)    7.2% (  -5% -
  22%)
                 Respell       19.68      (5.9%)       21.15      (6.6%)    7.5% (  -4% -
  21%)
               LowPhrase        0.46      (9.5%)        0.50     (10.2%)    7.6% ( -11% -
  30%)
                 Prefix3        5.25      (8.2%)        5.66      (7.7%)    7.9% (  -7% -
  25%)
        HighSloppyPhrase        1.54      (8.0%)        1.67     (13.1%)    8.5% ( -11% -
  32%)
             MedSpanNear        5.25      (7.0%)        5.72      (8.2%)    9.0% (  -5% -
  25%)
                Wildcard       12.44      (5.7%)       13.59      (6.5%)    9.2% (  -2% -
  22%)
         MedSloppyPhrase        2.27      (7.2%)        2.49      (8.5%)    9.5% (  -5% -
  27%)
                 MedTerm       28.16     (10.3%)       30.89      (9.9%)    9.7% (  -9% -
  33%)
                  Fuzzy1       18.91      (6.0%)       20.82      (6.7%)   10.1% (  -2% -
  24%)
                  Fuzzy2       19.69      (6.6%)       21.68      (7.5%)   10.1% (  -3% -
  25%)
              AndHighMed        7.79      (7.5%)        8.58      (6.1%)   10.1% (  -3% -
  25%)
             LowSpanNear        1.45      (5.7%)        1.60      (9.3%)   10.5% (  -4% -
  27%)
         LowSloppyPhrase       22.84      (7.7%)       25.45      (9.7%)   11.4% (  -5% -
  31%)
                 LowTerm       46.46      (6.8%)       52.90      (7.6%)   13.9% (   0% -
  30%)
              AndHighLow       35.92      (5.3%)       42.38      (7.1%)   18.0% (   5% -
  32%)
{code}

> Native MMapDir
> --------------
>
>                 Key: LUCENE-3178
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3178
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/store
>            Reporter: Michael McCandless
>              Labels: gsoc2014
>         Attachments: LUCENE-3178-Native-MMap-implementation.patch, LUCENE-3178-Native-MMap-implementation.patch,
LUCENE-3178-Native-MMap-implementation.patch
>
>
> Spinoff from LUCENE-2793.
> Just like we will create native Dir impl (UnixDirectory) to pass the right OS level IO
flags depending on the IOContext, we could in theory do something similar with MMapDir.
> The problem is MMap is apparently quite hairy... and to pass the flags the native code
would need to invoke mmap (I think?), unlike UnixDir where the code "only" has to open the
file handle.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message