lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Commented: (LUCENE-1410) PFOR implementation
Date Fri, 03 Oct 2008 17:26:44 GMT


Michael McCandless commented on LUCENE-1410:

bq. A: PFor.getNumFrameBits() does this for a given int[] at offset and size.

Super, I missed that -- I'll use it.

bq. Btw. after decompressing, the things are ints not vInts.

Oh yeah, I'll fix the prints in my test.

bq. There is some fallback code (decodeAnyFrame) that will decompress for any frame or reference

Right I was wanting the unrolled version to see the fastest result we can get for pfor, but
your next observation is spooky so maybe this isn't really going to help our test:

Btw. on my machine the unrolled 9 bit version is actually a bit slower than the loop, I don't
know why,
maybe there are too many buffer references (9) in the loop for the jit to cope with.
We should look at the asm that's produced?

bq. it's probably the JIT jumping in.

But strangely with -Xbatch I still see the effect.  And even stranger, on another machine
(Mac OS X) it gets *slower* when the JIT jumps in.  I'm spooked.

bq. Which block size did you use for PFor?

I used 128 but I'll try different sizes.

bq. For practical use, the start of each compressed block must be known, either from somewhere
else, or from the size of the previously encoded block.

But can I also get the "bytes consumed" when I ask decompress() to

When we really integrate, things like term infos and skipping data
will know how to jump to the start of blocks.  But for raw sequential
scanning, if the header is self-punctuating we would not need to store
the block size between each block.

> PFOR implementation
> -------------------
>                 Key: LUCENE-1410
>                 URL:
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Other
>            Reporter: Paul Elschot
>            Priority: Minor
>         Attachments: LUCENE-1410b.patch,
>   Original Estimate: 21840h
>  Remaining Estimate: 21840h
> Implementation of Patched Frame of Reference.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message