hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9413) Introduce common utils for File#setReadable/Writable/Executable and File#canRead/Write/Execute that work cross-platform
Date Mon, 25 Mar 2013 02:53:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13612362#comment-13612362
] 

Ivan Mitic commented on HADOOP-9413:
------------------------------------

Thanks Chris!

bq. I've spotted another instance of this problem, in YARN nodemanager NodeHealthScriptRunner#shouldRun,
which checks File#canExecute, and we've already seen that this is unreliable on Windows. The
problem manifests as a test failure in TestNodeHealthService.
I created YARN-506 and attached a patch that makes the appropriate changes in the yarn codebase.

Btw, I noticed a single place in the mapreduce project which calls File#canExecute. Will file
a separate Jira for this.

bq. 1. At the moment, I did not expose File#set* APIs that accept the ownerOnly parameter.
Adding this is strait forward, the only question is whether we need it. I didn't find use-cases
in Common and HDFS projects, but I've seen some in Yarn (only on one or two places, and those
could be changed to use chmod instead).
To also respond to my earlier comment from above. I don't think we need to expose these APIs,
at least not yet. I found only a single occurrence where these APIs are used, and using chmod
as a replacement seems appropriate. 
                
> Introduce common utils for File#setReadable/Writable/Executable and File#canRead/Write/Execute
that work cross-platform
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-9413
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9413
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>             Fix For: 3.0.0
>
>         Attachments: HADOOP-9413.commonfileutils.patch
>
>
> So far, we've seen many unittest and product bugs in Hadoop on Windows because Java's
APIs that manipulate with permissions do not work as expected. We've addressed many of these
problems on one-by-one basis (by either changing code a bit or disabling the test). While
debugging the remaining unittest failures we continue to run into the same patterns of problems,
and instead of addressing them one-by-one, I propose that we expose a set of equivalent wrapper
APIs that will work well for all platforms.
> Scanning thru the codebase, this will actually be a simple change as there are very few
places that use File#setReadable/Writable/Executable and File#canRead/Write/Execute (5 files
in Common, 9 files in HDFS).
> HADOOP-8973 contains additional context on the problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message