hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13651) S3Guard: S3AFileSystem Integration with MetadataStore
Date Tue, 01 Nov 2016 12:18:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625286#comment-15625286
] 

Steve Loughran commented on HADOOP-13651:
-----------------------------------------

bq. What needs to be done before we can commit this patch (besides the LOG.isDebugEnabled
thing)? 

Well, it is just a branch, so it's not expected to be perfect.

What will happen is once the merge is in, the branch is going to have to be kept in sync with
a piece of code which still has a major rate of change. Someone is going to have to volunteer
to do that regularly, or share the workload. I'd recommend a process of:

# checkout and build trunk
# run the hadoop-aws integration tests on trunk
# if the tests all worked, checkout this branch, rebase onto it, rerun those integration tests
# if the tests didn't work: and they used to: consider reporting a problem, ideally after
checking to see if something hasn't been opened already/your test setup hasn't regressed.

Step 2 makes sure that you don't rebase on broken code, and so get confused that test failures
are your fault.

I'd recommend doing this weekly, or if there's been a big change gone into s3a. It's already
time to do this, given the current difference between the branch and trunk

> S3Guard: S3AFileSystem Integration with MetadataStore
> -----------------------------------------------------
>
>                 Key: HADOOP-13651
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13651
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Aaron Fabbri
>            Assignee: Aaron Fabbri
>         Attachments: HADOOP-13651-HADOOP-13345.001.patch, HADOOP-13651-HADOOP-13345.002.patch,
HADOOP-13651-HADOOP-13345.003.patch
>
>
> Modify S3AFileSystem et al. to optionally use a MetadataStore for metadata consistency
and caching.
> Implementation should have minimal overhead when no MetadataStore is configured.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message