hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13760) S3Guard: add delete tracking
Date Wed, 24 May 2017 17:52:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16023325#comment-16023325
] 

Aaron Fabbri commented on HADOOP-13760:
---------------------------------------

Thanks for the updated patch [~mackrorysd].  Taking a look now.

{quote}
What do you think about eliminating deleteMetadata (for now - maybe it'll be needed for some
feature in the future), adding the option to destroy to just clean out the table but leave
the version marker and table itself, and then only using that function call on tests where
it matters?
{quote}
I'd prefer eliminating deleteMetadata(), and adding a boolean hint "destroyResources" to destroy().
 We can change DDB to not delete the table when destroyResources is false, and then use that
for test cleanup.  Just biased to keeping MetadataStore as simple as possible.

I'd also be ok with renaming existing deleteMetadata() to clearMetadata() or forgetMetadata()
/ forgetPath() and clearly marking it as for testing only.

> S3Guard: add delete tracking
> ----------------------------
>
>                 Key: HADOOP-13760
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13760
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Aaron Fabbri
>            Assignee: Sean Mackrory
>         Attachments: HADOOP-13760-HADOOP-13345.001.patch, HADOOP-13760-HADOOP-13345.002.patch,
HADOOP-13760-HADOOP-13345.003.patch, HADOOP-13760-HADOOP-13345.004.patch, HADOOP-13760-HADOOP-13345.005.patch,
HADOOP-13760-HADOOP-13345.006.patch, HADOOP-13760-HADOOP-13345.007.patch, HADOOP-13760-HADOOP-13345.008.patch,
HADOOP-13760-HADOOP-13345.009.patch, HADOOP-13760-HADOOP-13345.010.patch
>
>
> Following the S3AFileSystem integration patch in HADOOP-13651, we need to add delete
tracking.
> Current behavior on delete is to remove the metadata from the MetadataStore.  To make
deletes consistent, we need to add a {{isDeleted}} flag to {{PathMetadata}} and check it when
returning results from functions like {{getFileStatus()}} and {{listStatus()}}.  In HADOOP-13651,
I added TODO comments in most of the places these new conditions are needed.  The work does
not look too bad.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message