lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5722) Speed up MMapDirectory.seek()
Date Mon, 02 Jun 2014 13:34:02 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015375#comment-14015375
] 

Robert Muir commented on LUCENE-5722:
-------------------------------------

{quote}
For reference, this is the optimization I had in mind. I don't know if it helps for the multi-buffer
case, but may be worth a try.

The patch may not apply cleanly, its just for demonstartion purposes.
{quote}

I tested this with sorting on 1M and 10M wikipedia index: its a consistent 7% improvement.
+1 to just commit that one, and lets keep iterating on the more complex refactor!

> Speed up MMapDirectory.seek()
> -----------------------------
>
>                 Key: LUCENE-5722
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5722
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Robert Muir
>         Attachments: LUCENE-5722-multiseek.patch, LUCENE-5722.patch, LUCENE-5722.patch,
LUCENE-5722.patch, LUCENE-5722.patch
>
>
> For traditional lucene access which is mostly sequential, occasional advance(), I think
this method gets drowned out in noise.
> But for access like docvalues, its important. Unfortunately seek() is complex today because
of mapping multiple buffers.
> However, the very common case is that only one map is used for a given clone or slice.
> When there is the possibility to use only a single mapped buffer, we should instead take
advantage of ByteBuffer.slice(), which will adjust the internal mmap address and remove the
offset calculation. furthermore we don't need the shift/mask or even the negative check, as
they are then all handled with the ByteBuffer api: seek is a one-liner (with try/catch of
course to convert exceptions).
> This makes docvalues access 20% faster, I havent tested conjunctions or anyhting like
that.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message