hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9261) S3n filesystem can move a directory under itself -and so lose data
Date Fri, 01 Feb 2013 23:28:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569241#comment-13569241

Steve Loughran commented on HADOOP-9261:

This turns out to be harder than expected because of the requirement for POSIX return codes.

#rename file->file = true
#rename dir->dir = false
Also, we need to check the destination after the full destination is calculated, not upfront,
so that when the full pathname is calculated as parent+filename, the equality check needs
to be made then. 
# rename (path/file,path) returns true (shortcutting the copy)
# rename (path/dir, path) fails 

> S3n filesystem can move a directory under itself -and so lose data
> ------------------------------------------------------------------
>                 Key: HADOOP-9261
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9261
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 1.1.1, 2.0.2-alpha
>         Environment: Testing against S3 bucket stored on US West (Read after Write consistency;
eventual for read-after-delete or write-after-write)
>            Reporter: Steve Loughran
>         Attachments: HADOOP-9261.patch
> The S3N filesystem {{rename()}} doesn't make sure that the destination directory is not
a child or other descendant of the source directory. The files are copied to the new destination,
then the source directory is recursively deleted, so losing data.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message