hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15430) hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard
Date Thu, 03 May 2018 17:20:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462814#comment-16462814

Steve Loughran commented on HADOOP-15430:

Got this

# If you create a path with {{new Path(String)}}, then {{Path.normalizePath()}} with strip
trailing slashes off the path.
# If you create a path with {{new Path(URI)}} then the URI goes in as is. So if it is not
normalized, it stays un-normalized. And as downstream things creating new paths always use
that {{new Path(otherpath.getURI())}}, that propagates through things like qualify().

This is a fairly fundamental issue, and there's no way I'd want to go near path or FsShell.
Also explains why we didn't pick this up in the contract tests, we've been constructing paths
with the string ctor, not via a URI. Who knew?

Proposed, we do that stripping, and fix the tests where they fail.

> hadoop fs -mkdir -p path-ending-with-slash/ fails with s3guard
> --------------------------------------------------------------
>                 Key: HADOOP-15430
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15430
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.1.0
>            Reporter: Steve Loughran
>            Priority: Blocker
>         Attachments: HADOOP-15430-001.patch, HADOOP-15430-002.patch
> if you call {{hadoop fs -mkdir -p path/}} on the command line with a path ending in "/:.
you get a DDB error "An AttributeValue may not contain an empty string"

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message