hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10372) Deprecate S3 implementation.
Date Fri, 28 Feb 2014 10:07:22 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13915621#comment-13915621

Steve Loughran commented on HADOOP-10372:

it implements an inode structure atop s3, therefore can perform atomic directory renames and
deletes. The need for s3 and swifts blobstore clients to implement these bulk operations on
a per-file basis can create race conditions in a cluster (alongside the existing consistency
issues), which even s3:// cannot address. This cannot be corrected in a blobstore that does
not offer these operations as atomic actions. 

Have a look at HADOOP-9361 to see more on what we really expect a filesystem to do -including
contract tests that work with s3 as well as s3n

> Deprecate S3 implementation.
> ----------------------------
>                 Key: HADOOP-10372
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10372
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>            Reporter: Amandeep Khurana
>            Priority: Minor
> We encourage users to use the S3N implementation. We should consider deprecating the
older S3 implementation to avoid confusion down the road.

This message was sent by Atlassian JIRA

View raw message