hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10372) Deprecate S3 implementation.
Date Fri, 28 Feb 2014 10:07:22 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13915621#comment-13915621
] 

Steve Loughran commented on HADOOP-10372:
-----------------------------------------

it implements an inode structure atop s3, therefore can perform atomic directory renames and
deletes. The need for s3 and swifts blobstore clients to implement these bulk operations on
a per-file basis can create race conditions in a cluster (alongside the existing consistency
issues), which even s3:// cannot address. This cannot be corrected in a blobstore that does
not offer these operations as atomic actions. 

Have a look at HADOOP-9361 to see more on what we really expect a filesystem to do -including
contract tests that work with s3 as well as s3n

> Deprecate S3 implementation.
> ----------------------------
>
>                 Key: HADOOP-10372
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10372
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>            Reporter: Amandeep Khurana
>            Priority: Minor
>
> We encourage users to use the S3N implementation. We should consider deprecating the
older S3 implementation to avoid confusion down the road.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message