hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16138) hadoop fs mkdir / of nonexistent abfs container raises NPE
Date Fri, 22 Feb 2019 16:18:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775325#comment-16775325

Steve Loughran commented on HADOOP-16138:

If you try to create a subdir then it fails more meaningfully
2019-02-22 16:05:43,642 [main] DEBUG services.AbfsClient (AbfsRestOperation.java:executeHttpOperation(202))
- HttpRequest: 404,,cid=ba8b4d49-e05f-493e-97f4-bd1bde48f845,rid=f6ca5c36-f01f-002c-30c8-cad834000000,connMs=0,sendMs=0,recvMs=23,sent=0,recv=0,HEAD,https://abfswales1.dfs.core.windows.net/container2//?upn=false&action=getAccessControl&timeout=90
mkdir: `abfs://container2@abfswales1.dfs.core.windows.net/': No such file or directory
2019-02-22 16:05:43,644 [shutdown-hook-0] DEBUG azurebfs.AzureBlobFileSystem (AzureBlobFileSystem.java:close(427))
- AzureBlobFileSystem.close

One thing to consider here is FNFE the right thing to raise when looking for the root directory
-as its possible that this is being caught during mkdir as "we should create something". 
Maybe if the error handling can pick up the `x-ms-error-code=FilesystemNotFound` error, it
could generate a different response

FWIW, a call of "hadoop fs -mkdir s3a://hwdev-steve-new2/" does fail
mkdir: Bucket hwdev-steve-new2 does not exist, 
-the existence check in S3AFS is just throwing an FNFE, for mkdir to handle. 

Also, "hadoop fs mkdir" is clearly broken when it is told to create the root directory. 

Maybe both the stores would benefit from a new "ObjectStoreDoesNotExist" exception (Extends
FNFE) for a clearer message; we could even think about having FS shell handle it specially

> hadoop fs mkdir / of nonexistent abfs container raises NPE
> ----------------------------------------------------------
>                 Key: HADOOP-16138
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16138
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/azure
>    Affects Versions: 3.2.0
>            Reporter: Steve Loughran
>            Priority: Minor
> If you try to do a mkdir on the root of a nonexistent container, you get an NPE
> {code}
> hadoop fs -mkdir  abfs://container2@abfswales1.dfs.core.windows.net/  
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message