hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7973) DistributedFileSystem close has severe consequences
Date Fri, 13 Jan 2012 20:26:40 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13185825#comment-13185825

Daryn Sharp commented on HADOOP-7973:

The problem appears to be a case of two wrongs make a right.  Fixing a bug in HDFS-2450 exposed
another bug.  Unlike other filesystems, DFS used to strip the default port from its uris.
{{FileSystem.get("hdfs://host:port").getUri()}} did not return "hdfs://host:port".  It returned

The result is that MR must be getting another filesystem based on {{getUri()}} of one filesystem.
 When the port is stripped, two distinct filesystems are created.  It's a complete fluke that
MR worked because of the former bug.

So is the finalizer approach a no-go, or do you have another solution in mind?  As you mentioned
it's a public api, so is it better to "fix" the public api, or tell users don't use the public
> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>                 Key: HADOOP-7973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7973
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-7973.patch
> The way {{FileSystem#close}} works is very problematic.  Since the {{FileSystems}} are
cached, any {{close}} by any caller will cause problems for every other reference to it. 
Will add more detail in the comments.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message