lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-7323) Compound file writing should verify checksum of its sub-files
Date Wed, 08 Jun 2016 22:08:20 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-7323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael McCandless updated LUCENE-7323:
---------------------------------------
    Attachment: LUCENE-7323.patch

Patch, I think it's close.  It fixes our default
Lucene50CompoundFileFormat to verify the checksum of its sub-files
when writing.

I also had to close up external access to SimpleText's doc values and
postings format, i.e., you must use them only via SimpleTextCodec,
because these files (intentionally) don't write codec headers and
footers so you can't put them into a "normal" CFS file (SimpleText has
its own CFS that doesn't verify checksums).

I also made {{CodecUtil.read/writeCRC}} package private: do they
really need to be public?


> Compound file writing should verify checksum of its sub-files
> -------------------------------------------------------------
>
>                 Key: LUCENE-7323
>                 URL: https://issues.apache.org/jira/browse/LUCENE-7323
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>             Fix For: master (7.0), 6.2
>
>         Attachments: LUCENE-7323.patch
>
>
> For larger segments, there is a non-trivial window, from when IW
> writes sub-files, to when it then builds the CFS, during which the
> files can become corrupted (from external process, bad filesystem,
> hardware, etc.)
> Today we quietly build the CFS even if the sub-files are corrupted,
> but we can easily detect it, letting users catch corruption earlier
> (write time instead of read time).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message