hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9357) Fallback to default authority if not specified in FileContext
Date Wed, 27 Mar 2013 01:17:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13614779#comment-13614779
] 

Andrew Wang commented on HADOOP-9357:
-------------------------------------

Hi Eli, thanks for the review,

bq. The new code can live in the first case of the existing if right? Ie all URIs that need
fixing are absolute.

I thought about this a bit, and I think you can have URIs with a scheme and relative path,
e.g. {{hdfs:foo/bar}}, which should then just be treated as {{foo/bar}}. I added a test case
for this.

bq. More descriptive name than "fixPath"?

Since "fixRelativePartAndSchemaNoAuthority" doesn't roll off the tongue so well, I just changed
it back to the original "fixRelativePart". In FileSystem, this is done in checkPath (not a
great name either).

bq. Consider adding new Path method (similar to isAbsoluteAndSchemeAuthorityNull)

Good idea, added. I didn't refactor FileSystem's handling to use this since it's kind of gnarly,
but I could if desirable.

On putting this in the constructor, I don't think we can easily pursue that path. We can't
do the fixup until we know the default scheme of the filesystem where the path is being resolved,
which would mean passing un-fixed URIs around as a {{URI}} or {{String}} or {{NewPath}} before
we can make it into a {{Path}}. FS and FC already process incoming paths in fixup/check methods,
so that's probably the right place for this too.
                
> Fallback to default authority if not specified in FileContext
> -------------------------------------------------------------
>
>                 Key: HADOOP-9357
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9357
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>            Priority: Minor
>             Fix For: 3.0.0
>
>         Attachments: hadoop-9357-1.patch, hadoop-9357-2.patch, hadoop-9357-3.patch
>
>
> Currently, FileContext adheres rather strictly to RFC2396 when it comes to parsing absolute
URIs (URIs with a scheme). If a user asks for a URI like "hdfs:///tmp", FileContext will error
while FileSystem will add the authority of the default FS (e.g. turn it into "hdfs://defaultNN:port/tmp").

> This is technically correct, but FileSystem's behavior is nicer for users and okay based
on 5.2.3 in the RFC, so lets do it in FileContext too:
> {noformat}
> For backwards
> compatibility, an implementation may work around such references
> by removing the scheme if it matches that of the base URI and the
> scheme is known to always use the  syntax.  The parser
> can then continue with the steps below for the remainder of the
> reference components.  Validating parsers should mark such a
> misformed relative reference as an error.
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message