hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16138) hadoop fs mkdir of nonexistent abfs container raises NPE
Date Fri, 22 Feb 2019 16:04:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775313#comment-16775313
] 

Steve Loughran commented on HADOOP-16138:
-----------------------------------------

{code}
2019-02-22 16:00:05,804 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:initialize(101))
- Initializing AzureBlobFileSystem for abfs://container2@abfswales1.dfs.core.windows.net/
2019-02-22 16:00:06,082 [main] DEBUG services.AbfsClientThrottlingIntercept (AbfsClientThrottlingIntercept.java:initializeSingleton(62))
- Client-side throttling is enabled for the ABFS file system.
2019-02-22 16:00:06,102 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:getFileStatus(434))
- AzureBlobFileSystem.getFileStatus path: abfs://container2@abfswales1.dfs.core.windows.net/
2019-02-22 16:00:06,102 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:performAbfsAuthCheck(1101))
- ABFS authorizer is not initialized. No authorization check will be performed.
2019-02-22 16:00:06,102 [main] DEBUG azurebfs.AzureBlobFileSystemStore (AzureBlobFileSystemStore.java:getIsNamespaceEnabled(200))
- Get root ACL status
2019-02-22 16:00:06,132 [main] DEBUG services.AbfsClient (AbfsRestOperation.java:executeHttpOperation(152))
- Signing request with shared key
2019-02-22 16:00:06,136 [main] DEBUG services.SharedKeyCredentials (SharedKeyCredentials.java:signRequest(96))
- Signing request with timestamp of Fri, 22 Feb 2019 16:00:06 GMT and signature SharedKey
/* snip */
2019-02-22 16:00:06,136 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51))
- Response Headers
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   Accept-Charset=utf-8
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   x-ms-version=2018-11-09
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   Accept=application/json, application/octet-stream
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   x-ms-date=Fri, 22 Feb 2019 16:00:06 GMT
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   User-Agent=Azure Blob FS/3.3.0-SNAPSHOT (JavaJRE 1.8.0_121; MacOSX 10.14; openssl-1.0)
unknown
2019-02-22 16:00:06,137 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   x-ms-client-request-id=d4484925-a273-482e-8aa9-e0fdef6cdf82
2019-02-22 16:00:06,138 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   Content-Type=
2019-02-22 16:00:06,277 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(51))
- Request Headers
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   HTTP Response=HTTP/1.1 404 The specified filesystem does not exist.
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   x-ms-version=2018-11-09
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   Server=Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   x-ms-error-code=FilesystemNotFound
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   x-ms-request-id=d4459a80-901f-0015-41c7-ca9890000000
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsIoUtils (AbfsIoUtils.java:dumpHeadersToDebugLog(54))
-   Date=Fri, 22 Feb 2019 16:00:05 GMT
2019-02-22 16:00:06,279 [main] DEBUG services.AbfsClient (AbfsRestOperation.java:executeHttpOperation(202))
- HttpRequest: 404,,cid=d4484925-a273-482e-8aa9-e0fdef6cdf82,rid=d4459a80-901f-0015-41c7-ca9890000000,connMs=26,sendMs=0,recvMs=138,sent=0,recv=0,HEAD,https://abfswales1.dfs.core.windows.net/container2//?upn=false&action=getAccessControl&timeout=90
2019-02-22 16:00:06,285 [main] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:getFileStatus(434))
- AzureBlobFileSystem.getFileStatus path: null
2019-02-22 16:00:06,286 [main] DEBUG fs.FsShell (FsShell.java:run(344)) - Error
java.lang.NullPointerException
	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:712)
	at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:602)
	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.makeQualified(AzureBlobFileSystem.java:466)
	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:436)
	at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1695)
	at org.apache.hadoop.fs.shell.Mkdir.processNonexistentPath(Mkdir.java:76)
	at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
	at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:270)
	at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
	at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
	at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
	at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
-mkdir: Fatal internal error
java.lang.NullPointerException
	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:712)
	at org.apache.hadoop.fs.FileSystem.makeQualified(FileSystem.java:602)
	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.makeQualified(AzureBlobFileSystem.java:466)
	at org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.getFileStatus(AzureBlobFileSystem.java:436)
	at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1695)
	at org.apache.hadoop.fs.shell.Mkdir.processNonexistentPath(Mkdir.java:76)
	at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
	at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:270)
	at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
	at org.apache.hadoop.fs.shell.Command.run(Command.java:177)
	at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
	at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
2019-02-22 16:00:06,290 [shutdown-hook-0] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:close(427))
- AzureBlobFileSystem.close
{code}

> hadoop fs mkdir of nonexistent abfs container raises NPE
> --------------------------------------------------------
>
>                 Key: HADOOP-16138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16138
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Priority: Minor
>
> If you try to do a mkdir on the root of a nonexistent container, you get an NPE
> {code}
> hadoop fs -mkdir  abfs://container2@abfswales1.dfs.core.windows.net/  
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message