hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-3518) Want NIO.2 (JSR 203) file system provider for Hadoop FileSystem
Date Wed, 27 Apr 2016 10:10:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15259913#comment-15259913

Steve Loughran commented on HADOOP-3518:

[~damiencarol] -hi, I hadn't seen this before, sorry if you felt ignored.

# Moving to NIO can only be good as a drop in for RawLocalFileSystem; HADOOP-9590 proposed
upgrading that, maybe having a RawNioFileSystem would be safer.
# This would really help Windows. But that also means windows testing. Are you set up for
that? If not, you'll need help
# Testing really matters for filesystems. That's why we have both a filesystem specification
and a large set of FS contract tests. These are all the foundational tests ... they verify
the core API works, though they don't check things like concurrency, consistency, durability
& failure handling ...

> Want NIO.2 (JSR 203) file system provider for Hadoop FileSystem
> ---------------------------------------------------------------
>                 Key: HADOOP-3518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3518
>             Project: Hadoop Common
>          Issue Type: Wish
>          Components: fs
>            Reporter: Tom White
> JSR 203 (aka "NIO.2" or "more NIO") is defining a rich set of classes for interacting
with files and file systems (as well as other NIO enhancements). It is scheduled to be released
as a part of Java 7.
> This motivation behind this issue is to see if NIO.2 can be used as an interface to Hadoop's
FileSystem class before NIO.2 is finalized, thus giving Hadoop developers an opportunity to
influence NIO's design (if necessary). Also, learning more about NIO.2 may inform design decisions
for Hadoop filesystems.
> The starting point for this work should be the java.nio.file.spi package (http://openjdk.java.net/projects/nio/javadoc/java/nio/file/spi/package-summary.html).
There is an example of a filesystem provider (for ZIP files) linked from the OpenJDK page
for NIO.2: http://openjdk.java.net/projects/nio/. This page also has other useful links, such
as a JavaOne talk, javadoc and source code.

This message was sent by Atlassian JIRA

View raw message