hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7973) DistributedFileSystem close has severe consequences
Date Thu, 19 Jan 2012 23:40:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189496#comment-13189496

Daryn Sharp commented on HADOOP-7973:

I don't recall the jira but it was quite a few years old and very lively.  Last week I spoke
to Suresh about ref counts and he did have some legitimate concerns.  After further reflection,
fs.close also closes open streams and that behavior might be expected in some cases.  Unbalanced
opens & closes would lead to situations where a close, that formerly caused all fs streams
to close, would be deferred into the future (if ever).

I'm fine with revisiting ref counts or a more comprehensive handling of close.  In this case,
my suggestion would be on another jira targeted to trunk.  I've scaled back the "fix" to be
as minimal as possible, and not alter any existing behaviors.  Yes, it's a bit of a hack,
but I'm paranoid of breaking something on 205/1.x and this bug is slowing production deployment.

> DistributedFileSystem close has severe consequences
> ---------------------------------------------------
>                 Key: HADOOP-7973
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7973
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 1.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>         Attachments: HADOOP-7973-2.patch, HADOOP-7973-3.patch, HADOOP-7973-4.patch, HADOOP-7973.patch
> The way {{FileSystem#close}} works is very problematic.  Since the {{FileSystems}} are
cached, any {{close}} by any caller will cause problems for every other reference to it. 
Will add more detail in the comments.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message