hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14913) Sticky bit implementation for Rename operation in Azure fs
Date Wed, 11 Oct 2017 14:48:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Steve Loughran updated HADOOP-14913:
    Attachment: HADOOP-14913-004.patch

Patch 004; testing azure ireland. If Yetus is happy this is what I'll commit.

bq. So accordingly, instead of throwing FNFE, I am returning false. This is same behaviour
as Auth-not-enabled case too in the rename implementation.

Sorry I'd missed that. Not generally the normal behaviour, and another example of why rename
true/false is such a weak form of signalling an error. We should really get HADOOP-11452 finished
& encourage code to move to it, at least in our committer logic. The changes in patch
003 line up for this better, because rename/3 can just throw the exceptions raised; no need
for catch & return false logic.

bq. If your concern is auth enabling is not testing whole paths of all fs calls, may be we
should consider having contract test runs with auth enabled cases. However it must be a seperate
endeavour unrelated to this change('Rename') alone.

yeah, it's a bigger change. I've wondered if there is an easy way to do it, but the answer
is "no", because of all the setup/teardown of permissions. You're going to need to run your
own tests downstream I'm afraid.

h3. changes to production code. 

Minor cleanup to logging and exceptions in {{performStickyBitCheckForRenameOperation}}: no
need for toString() in debug calls (potentially expensive) or in exceptions. Switch for srcKey
to srcPath in exception text.

h3. Test changes

* in all the new code I've made the use of ContractTestUtils static; that keeps the code leaner.
* added a new operation {{deleteRecursive}} which combines access right setup with the delete,
and downgrades failures to log @ error...this stops it ever hiding the inner exceptions from
the try/catch. Using in all the new/modified tests, but not the existing ones
* {{testRenameOnNonExistentSourceWithStickyBit}}:  dest dir {{parentDstDir}} now different
source dir; fix up assertions before rename() to be consistent.
* use {{final}} for variables used in doAs() to make backport to java 7 straightforward,.
* javadoc tuning.

Given all the assert true/false on renames, I also added a new ContractTestUtils operation
for this,{{ContractTestUtils.assertRenameOutcome(fs, srcPath, dstPath, expected)}}.  This
does the rename and if outcome != expected throws an assertion with the rename src and dest
on failure. Don't be afraid to add new operations there if they are broadly applicable across

> Sticky bit implementation for Rename operation in Azure fs
> ----------------------------------------------------------
>                 Key: HADOOP-14913
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14913
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs, fs/azure
>            Reporter: Varada Hemeswari
>            Assignee: Varada Hemeswari
>              Labels: azure, fs, secure
>         Attachments: HADOOP-14193.001.patch, HADOOP-14193.002.patch, HADOOP-14913-003.patch,
> When authorization is enabled in WASB filesystem, there is a need for stickybit in cases
where multiple users can create files under a shared directory. This additional check for
sticky bit is reqired since any user can delete/rename another user's file because the parent
has WRITE permission for all users.
> The purpose of this jira is to implement sticky bit equivalent for 'rename' call when
authorization is enabled.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message