lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5570) FSDirectory's fsync() is lenient
Date Thu, 03 Apr 2014 19:23:17 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13959110#comment-13959110
] 

Michael McCandless commented on LUCENE-5570:
--------------------------------------------

bq. if "something bad happens" (e.g. two indexwriters/dirs on the same path, or some other
shenanigans), and the file is actually in stale files, but was say actually deleted on the
filesystem, the underlying fsync() call will create a new 0-byte file and fsync that.

This is truly awful: sync of a non-existent file should not bring a 0-byte file into existence!

bq. it tracks 'stale files' as it writes (this seems pointless), and only actually fsyncs
the intersection of that 'stale files' and the filenames passed as argument to sync(). So
any bogus names passed to sync() are just silently ignored

This is because IW passes all files referenced by all segments when it commits(), i.e. we
push the responsibility of remembering which files are written but not sync'd down to Directory.
 This used to be IW's responsibility, but we changed that in LUCENE-2328 I think.

> FSDirectory's fsync() is lenient
> --------------------------------
>
>                 Key: LUCENE-5570
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5570
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/store
>            Reporter: Robert Muir
>         Attachments: LUCENE-5570.patch
>
>
> This method has a lot of problems:
> 1. it tracks 'stale files' as it writes (this seems pointless), and only actually fsyncs
the intersection of that 'stale files' and the filenames passed as argument to sync(). So
any bogus names passed to sync() are just silently ignored
> 2. if "something bad happens" (e.g. two indexwriters/dirs on the same path, or some other
shenanigans), and the file is actually in stale files, but was say actually deleted on the
filesystem, the underlying fsync() call will create a new 0-byte file and fsync that.
> In my opinion we should do none of this. we should throw exceptions when this stuff is
wrong.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message