hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14468) S3Guard: make short-circuit getFileStatus() configurable
Date Mon, 05 Feb 2018 22:28:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353020#comment-16353020
] 

Steve Loughran commented on HADOOP-14468:
-----------------------------------------

One thing related to this is whether we should have a TTL on tombstone markers.

Even in non-auth mode, when we reconcile the listings, files recorded as deleted are omitted.
If someone can create that file via another client, will it ever be seen in listings?

> S3Guard: make short-circuit getFileStatus() configurable
> --------------------------------------------------------
>
>                 Key: HADOOP-14468
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14468
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1
>            Reporter: Aaron Fabbri
>            Assignee: Aaron Fabbri
>            Priority: Minor
>
> Currently, when S3Guard is enabled, getFileStatus() will skip S3 if it gets a result
from the MetadataStore (e.g. dynamodb) first.
> I would like to add a new parameter {{fs.s3a.metadatastore.getfilestatus.authoritative}}
which, when true, keeps the current behavior.  When false, S3AFileSystem will check both S3
and the MetadataStore.
> I'm not sure yet if we want to have this behavior the same for all callers of getFileStatus(),
or if we only want to check both S3 and MetadataStore for some internal callers such as open().



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message