hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9507) LocalFileSystem rename() is broken in some cases when destination exists
Date Thu, 25 Jul 2013 22:43:49 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13720163#comment-13720163

Chris Nauroth commented on HADOOP-9507:

It is closest to the existing behavior on non-Windows platforms

Just to elaborate on this a bit, talking with Arpit made me realize that version 2 of the
patch (copy-delete) would have accidentally introduced new behavior on Linux.  Currently on
Linux, a rename crossing file systems would fail in {{File#renameTo}} and then fall through
to {{FileUtil#copy}}.  In version 2, we would instead trigger the copy-delete logic, which
would have a different result.

The goal of this issue was to provide a match to this method's existing behavior on platforms
where {{File#renameTo}} behaves differently, like Windows.  Therefore, I'm now in favor of
version 3 of the patch (delete-rename), not only for performance but also for correctness.

I'll provide new patches shortly to address Arpit's final comment.

> LocalFileSystem rename() is broken in some cases when destination exists
> ------------------------------------------------------------------------
>                 Key: HADOOP-9507
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9507
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 3.0.0, 1-win, 2.1.0-beta, 1.3.0
>            Reporter: Mostafa Elhemali
>            Assignee: Chris Nauroth
>            Priority: Minor
>         Attachments: HADOOP-9507-branch-1.1.patch, HADOOP-9507-branch-1.3.patch, HADOOP-9507-branch-1-win.1.patch,
HADOOP-9507-branch-1-win.3.patch, HADOOP-9507.branch-1-win.patch, HADOOP-9507-trunk.1.patch,
HADOOP-9507-trunk.2.patch, HADOOP-9507-trunk.3.patch
> The rename() method in RawLocalFileSystem uses FileUtil.copy() without realizing that
FileUtil.copy() has a special behavior that if you're copying /foo to /bar and /bar exists
and is a directory, it'll copy /foo inside /bar instead of overwriting it, which is not what
rename() wants. So you end up with weird behaviors like in this repro:
> {code}
> c:
> cd \
> md Foo
> md Bar
> md Foo\X
> md Bar\X
> hadoop fs -mv file:///c:/Foo file:///c:/Bar
> {code}
> At the end of this, you would expect to find only Bar\X, but you instead find Bar\X\X.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message