hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephen Chu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-9705) FsShell cp -p does not preserve directory attibutes
Date Mon, 08 Jul 2013 03:59:49 GMT
Stephen Chu created HADOOP-9705:
-----------------------------------

             Summary: FsShell cp -p does not preserve directory attibutes
                 Key: HADOOP-9705
                 URL: https://issues.apache.org/jira/browse/HADOOP-9705
             Project: Hadoop Common
          Issue Type: Bug
          Components: fs
    Affects Versions: 2.0.4-alpha, 3.0.0
            Reporter: Stephen Chu


HADOOP-9338 added the -p flag to preserve file attributes when copying.

However, cp -p does not preserve directory attributes. It'd be useful to add this functionality.

For example, the following shows that the modified time is not preserved
{code}
[schu@hdfs-snapshots-1 ~]$ $HADOOP_HOME/bin/hdfs dfs -mkdir /user/schu/testDir1
[schu@hdfs-snapshots-1 ~]$ $HADOOP_HOME/bin/hdfs dfs -ls /user/schu/
Found 1 items
drwxr-xr-x   - schu supergroup          0 2013-07-07 20:25 /user/schu/testDir1
[schu@hdfs-snapshots-1 ~]$ $HADOOP_HOME/bin/hdfs dfs -cp -p /user/schu/testDir1 /user/schu/testDir2
[schu@hdfs-snapshots-1 ~]$ $HADOOP_HOME/bin/hdfs dfs -ls /user/schu
Found 2 items
drwxr-xr-x   - schu supergroup          0 2013-07-07 20:25 /user/schu/testDir1
drwxr-xr-x   - schu supergroup          0 2013-07-07 20:35 /user/schu/testDir2
[schu@hdfs-snapshots-1 ~]$ 
{code}

The preserve logic is in CommandWithDestination#copyFileToTarget, which is only called with
files.

{code}
  protected void processPath(PathData src, PathData dst) throws IOException {
    if (src.stat.isSymlink()) {
      // TODO: remove when FileContext is supported, this needs to either                
                                                                                    
      // copy the symlink or deref the symlink                                           
                                                                                    
      throw new PathOperationException(src.toString());
    } else if (src.stat.isFile()) {
      copyFileToTarget(src, dst);
    } else if (src.stat.isDirectory() && !isRecursive()) {
      throw new PathIsDirectoryException(src.toString());
    }
  }
{code}

{code}
  /**                                                                                    
                                                                                    
   * Copies the source file to the target.                                               
                                                                                    
   * @param src item to copy                                                             
                                                                                    
   * @param target where to copy the item                                                
                                                                                    
   * @throws IOException if copy fails                                                   
                                                                                    
   */
  protected void copyFileToTarget(PathData src, PathData target) throws IOException {
    src.fs.setVerifyChecksum(verifyChecksum);
    if (src != null) {
      throw new PathExistsException("hi");
    }
    InputStream in = null;
    try {
      in = src.fs.open(src.path);
      copyStreamToTarget(in, target);
      if(preserve) {
        target.fs.setTimes(
          target.path,
          src.stat.getModificationTime(),
          src.stat.getAccessTime());
        target.fs.setOwner(
          target.path,
          src.stat.getOwner(),
          src.stat.getGroup());
        target.fs.setPermission(
          target.path,
          src.stat.getPermission());
        System.out.println("Preserving");

        if (src.fs.equals(target.fs)) {
            System.out.println("Same filesystems");
          src.fs.preserveAttributes(src.path, target.path);
        }
        throw new IOException("hi");
      }
    } finally {
      IOUtils.closeStream(in);
    }
  }
{code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message