hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akira AJISAKA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10392) Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem)
Date Fri, 08 Aug 2014 10:37:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090605#comment-14090605
] 

Akira AJISAKA commented on HADOOP-10392:
----------------------------------------

Thanks [~daryn] for the comment.
bq. Changing to localFs.makeQualified(path) is wrong because it will artificial fail for non-local
paths.
Hadoop Streaming '-file' option only accepts local files. If non-local path is set, StreamJob
will fail in
{code}
   validate(packageFiles_);
{code}
method, which validates if the file path exists in localFs.
{code}
  private void validate(final List<String> values)
  throws IllegalArgumentException {
    for (String file : values) {
      File f = new File(file);
      if (!FileUtil.canRead(f)) {
        fail("File: " + f.getAbsolutePath()
          + " does not exist, or is not readable.");
      }
    }
  }
{code}
Therefore, I'm thinking it's okay to fail artificially in {{LocalFs.makeQualified(path)}}
because the job eventually fails with non-local paths.
In addition, it's also fine for me to use {{Path.makeQualified(URI, Path)}} in StreamingJob,
that does not change the behavior at all.

> Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem)
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-10392
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10392
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 2.3.0
>            Reporter: Akira AJISAKA
>            Assignee: Akira AJISAKA
>            Priority: Minor
>              Labels: newbie
>         Attachments: HADOOP-10392.2.patch, HADOOP-10392.3.patch, HADOOP-10392.4.patch,
HADOOP-10392.4.patch, HADOOP-10392.5.patch, HADOOP-10392.patch
>
>
> There're some methods calling Path.makeQualified(FileSystem), which causes javac warning.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message