hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities to FS and FC to complement StreamCapabilities
Date Fri, 23 Aug 2019 19:53:06 GMT
steveloughran commented on a change in pull request #568: HADOOP-15691 Add PathCapabilities
to FS and FC to complement StreamCapabilities
URL: https://github.com/apache/hadoop/pull/568#discussion_r317280104
 
 

 ##########
 File path: hadoop-common-project/hadoop-common/src/site/markdown/filesystem/filesystem.md
 ##########
 @@ -1470,3 +1470,144 @@ hsync        | HSYNC      | Syncable         | Flush out the data
in client's us
 in:readahead | READAHEAD  | CanSetReadahead  | Set the readahead on the input stream.
 dropbehind   | DROPBEHIND | CanSetDropBehind | Drop the cache.
 in:unbuffer  | UNBUFFER   | CanUnbuffer      | Reduce the buffering on the input stream.
+
+## <a name="PathCapabilities"></a> interface `PathCapabilities`
+
+The `PathCapabilities` provides a way to programmatically query the operations
+offered by a FileSystem or FileContext instance under a given path.
+
+```java
+public interface PathCapabilities {
+  boolean hasPathCapability(Path path, String capability)
+      throws IOException;
+}
+```
+
+There are a number of goals here:
+
+1. Allow callers to probe for optional filesystem operations without actually
+having to invoke them.
+1. Allow filesystems with their own optional per-instance features to declare
+whether or not they are active for the specific instance.
+1. Allow for fileystem connectors which work with object stores to expose the
+fundamental difference in semantics of these stores (e.g: files not visible
+until closed, file rename being `O(data)`), directory rename being non-atomic,
+etc.
+
+### Available Capabilities
+
+Capabilities are defined with a store prefix and an arbitrary (but hopefully
+meaningful) string.
+
+All custom filesystem-specific capabilities MUST be given the prefix of that
+filesystem schema. The standard schemas are:
+
+* `fs` : File system capabilities
+* `object` : Object capabilities.
+
+The exact set of operations and their names are evolving.
+
+Consult the javadocs for `org.apache.hadoop.fs.PathCapabilities` for the
+standard set of capabilities.
+
+Individual filesystems may offer their own set of capabilities which
+can be probed for. These begin with the same prefix as the filesystem schema,
+such as `hdfs:` or `s3a:`.
+
+### `boolean hasPathCapability(path, capability)`
+
+Probe for a filesystem instance offering a specific capability under the
+given path.
+
+#### Postconditions
+
+```python
+if fs_supports_the_feature(path, capability):
+  return True
+else:
+  return False
+```
+
+Return: `True`, iff the specific capability is, the the best of the
 
 Review comment:
   reworked. Also moved the entire interface into its own file.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message