hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Palanisamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16247) NPE in FsUrlConnection
Date Wed, 01 May 2019 00:17:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830771#comment-16830771
] 

Karthik Palanisamy commented on HADOOP-16247:
---------------------------------------------

Yes [~daryn], old change will not work for HDFS fs url.  I have a new code change now. 

The problem initailly by #URL.getPath(), this will not work if URL file path have any space.
So HADOOP-15217 fix is to change URL.toURI() which handle space issue. Now, there is another
problem with URI. If URI contains relative/opaque file path, then path will be null in URI
object. 

I think only change that work for both file and hdfs schema is,

{code}

FsUrlConnection.java

if(uri.isOpaque() && uri.getScheme().equals("file")) {
 is = fs.open(new Path(url.toURI().getSchemeSpecificPart()));
 } else {
 is = fs.open(new Path(uri));
 }

{code}

 

Thanks [~bharatviswa] for offline discussion. 

[~daryn] [~jojochuang] Would you please suggest if any?

> NPE in FsUrlConnection
> ----------------------
>
>                 Key: HADOOP-16247
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16247
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: hdfs-client
>    Affects Versions: 3.1.2
>            Reporter: Karthik Palanisamy
>            Assignee: Karthik Palanisamy
>            Priority: Major
>         Attachments: HADOOP-16247-001.patch, HADOOP-16247-002.patch, HADOOP-16247-003.patch,
HADOOP-16247-004.patch, HADOOP-16247-005.patch
>
>
> FsUrlConnection doesn't handle relativePath correctly after the change [HADOOP-15217|https://issues.apache.org/jira/browse/HADOOP-15217]
> {code}
> Exception in thread "main" java.lang.NullPointerException
>  at org.apache.hadoop.fs.Path.isUriPathAbsolute(Path.java:385)
>  at org.apache.hadoop.fs.Path.isAbsolute(Path.java:395)
>  at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:87)
>  at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:636)
>  at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:930)
>  at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:631)
>  at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:454)
>  at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:146)
>  at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:347)
>  at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:899)
>  at org.apache.hadoop.fs.FsUrlConnection.connect(FsUrlConnection.java:62)
>  at org.apache.hadoop.fs.FsUrlConnection.getInputStream(FsUrlConnection.java:71)
>  at java.net.URL.openStream(URL.java:1045)
>  at UrlProblem.testRelativePath(UrlProblem.java:33)
>  at UrlProblem.main(UrlProblem.java:19)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message