lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] Updated: (LUCENE-1410) PFOR implementation
Date Sun, 05 Oct 2008 14:57:44 GMT


Michael McCandless updated LUCENE-1410:


Attached new

  - If you run it on _x.frq, it will split that into _x.frq.frq and
    _x.frq.doc so we can separately test frq vs docs

  - Added checksum.

  - Added zeroing of reused IntBuffer before calling PFor.compress
    (seems to be necessary?  Else I trip an assert inside PFor).

Paul, indeed I get a different checksum for vint vs pfor decoding.

 I think the bug is somewhere in pfor, I'm guessing in the exception
logic, because the difference I see is suddenly pfor returns 0 when it
should have returned a large int relative to the other ints nearby.

Maybe this is why exception processing looked so much faster :)

I'll hold off posting more perf results until we can resolve that.

To see the checksum run it with asserts, eg like this:

  java -ea oal.util.pfor.TestPFor2 /path/to/index _x.prx

It then prints out SUM lines after each iteration.

If you set DEBUG = true, it'll print the first 1000 values and then
search for "v=0".

> PFOR implementation
> -------------------
>                 Key: LUCENE-1410
>                 URL:
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Other
>            Reporter: Paul Elschot
>            Priority: Minor
>         Attachments: LUCENE-1410b.patch,,,
>   Original Estimate: 21840h
>  Remaining Estimate: 21840h
> Implementation of Patched Frame of Reference.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message