hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-6442) Better FileContext getFSoPath comment
Date Tue, 15 Dec 2009 23:01:19 GMT
Better FileContext getFSoPath comment

                 Key: HADOOP-6442
                 URL: https://issues.apache.org/jira/browse/HADOOP-6442
             Project: Hadoop Common
          Issue Type: Bug
            Reporter: Eli Collins

The getFSofPath method in FileContext returns a subclass of AbstractFileSystem that matches
the scheme of the given path (eg maps "hdfs" -> class Hdfs). However the "file" scheme
(eg "file:///xxx") maps to multiple classes:

    -> DelegateToFileSystem -> RawLocalFs
              -> RawLocalFileSystem
    -> FilterFs -> ChecksumFs -> LocalFs 

eg "file:/f" may have been created using RawLocalFileSystem but getFSofPath returns an instance
of LocalFs because core-default.xml maps 
"fs.AbstractFileSystem.*file*.impl" to "LocalFs". The current API can lead to surprises, eg
trying to delete (using FileContext) a file created using RawLocalFs doesn't use the appropriate
delete method because getFSoPath returns a LocalFs and so the delete method in ChecksumFs
is used. The latter doesn't delete dangling symlinks like the former which is how I noticed.
There could be other bugs, eg looking for checksums for a file that wasn't created with ChecksumFs.
Is there any reason to not unify local file systems under a single path in the class hierarchy?
Either way we should doc this behavior in a comment in getFSofPath.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message