hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mingliang Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14428) s3a: mkdir appears to be broken
Date Fri, 19 May 2017 01:17:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016762#comment-16016762
] 

Mingliang Liu commented on HADOOP-14428:
----------------------------------------

The idea in [HADOOP-14255] is to delete the unnecessary fake object for all its ancestors.
The idea makes sense if we start deleting unnecessary fake objects from current empty directory's
parent.
{code}
deleteUnnecessaryFakeDirectories(f.getParent());
{code}
However, with trailing slash, the {{Path::getParent()}} is not behaviouring what we expected:
{{s3a://my-bucket/dir/}}'s parent is {{s3a://my-bucket/dir}} instead of {{s3a://my-bucket}}.
The original patch got passing unit tests because we didn't add trailing '/' in the test path.
{code:title=Path::getParent()}

  /**
   * Returns the parent of a path or null if at root.
   * @return the parent of a path or null if at root
   */
  public Path getParent() {
    String path = uri.getPath();
    int lastSlash = path.lastIndexOf('/');
    int start = startPositionWithoutWindowsDrive(path);
    if ((path.length() == start) ||               // empty path
        (lastSlash == start && path.length() == start+1)) { // at root
      return null;
    }
    String parent;
    if (lastSlash==-1) {
      parent = CUR_DIR;
    } else {
      parent = path.substring(0, lastSlash==start?start+1:lastSlash);
    }
    return new Path(uri.getScheme(), uri.getAuthority(), parent);
  }
{code}

So we simply trusted {{Path::getParent()}} but it betrays us here. We may normalize (remove
the trailing '/') before we calls {{getParent}}, or we figure out the parent by ourselves.

> s3a: mkdir appears to be broken
> -------------------------------
>
>                 Key: HADOOP-14428
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14428
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/s3
>    Affects Versions: 3.0.0-alpha2, HADOOP-13345
>            Reporter: Aaron Fabbri
>            Priority: Blocker
>
> Reproduction is:
> hadoop fs -mkdir s3a://my-bucket/dir/
> hadoop fs -ls s3a://my-bucket/dir/
> ls: `s3a://my-bucket/dir/': No such file or directory
> I believe this is a regression from HADOOP-14255.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message