hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13345) S3Guard: Improved Consistency for S3A
Date Thu, 05 Jan 2017 12:01:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801188#comment-15801188

Lei (Eddy) Xu commented on HADOOP-13345:

Hi, [~fabbri], [~liuml07], [~stevel@apache.org].   Happy new year.

[~liuml07], regarding HADOOP-13650 (CLI),  for {{DynamoDBMetadataStore#initialize(Configuration)}},
does it still require s3 fs name defined in the configuration?  I think that the user might
not be used to specify {{fs.defaultFS}} to an S3 bucket. It seems that logic that creating
AWS credential from S3 URL is deeply in the code.  The consequence of it is that even {{hadoop
s3a [init|destroy]}} can directly use a DynamoDB URL to create or clear / destroy the metadata
store, it still requires the user to specify a s3a url {{s3a://bucket}} to create the FS instance.
 It is also undesirable for mutli-bucket cases, i.e., sharing one dynamodb table with multiple

Should we get the current form of CLI committed first, then we change the CLI parameters after
the feature branch merged into trunk?


> S3Guard: Improved Consistency for S3A
> -------------------------------------
>                 Key: HADOOP-13345
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13345
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs/s3
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>         Attachments: HADOOP-13345.prototype1.patch, S3C-ConsistentListingonS3-Design.pdf,
S3GuardImprovedConsistencyforS3A.pdf, S3GuardImprovedConsistencyforS3AV2.pdf, s3c.001.patch
> This issue proposes S3Guard, a new feature of S3A, to provide an option for a stronger
consistency model than what is currently offered.  The solution coordinates with a strongly
consistent external store to resolve inconsistencies caused by the S3 eventual consistency

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message