lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5578) Stored fields might accumulate checksums on merges
Date Fri, 04 Apr 2014 23:29:16 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13960784#comment-13960784
] 

Uwe Schindler commented on LUCENE-5578:
---------------------------------------

bq. But I would like to write a better fix that doesn't rely on the length of the file to
compute the end offset of the last chunk.

We already changed the stored fields index format because of the additional checkum. Maybe
we can add another entry at position {{maxDoc}} in the index file pointing to the data block
after the last document?

Personally I also don't like seeking to positions relative to the end of the file.

> Stored fields might accumulate checksums on merges
> --------------------------------------------------
>
>                 Key: LUCENE-5578
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5578
>             Project: Lucene - Core
>          Issue Type: Bug
>            Reporter: Adrien Grand
>            Assignee: Adrien Grand
>            Priority: Blocker
>             Fix For: 4.8
>
>         Attachments: LUCENE-5578.patch
>
>
> The bulk merge operation of our stored fields format is optimized in order to avoid decompressing
data when not needed. In order to know the offset of the end of the current block, it either
consults the stored fields index, or uses {{fieldsStream.length()}} for the last chunk.
> However, we just added checksums at the end of index files, so it might currently copy
the current checksum in addition to the last chunk, and then write a new checksum.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message