hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Hale (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6414) Distcp command very slow to enumerate files needing
Date Tue, 23 Jun 2015 20:29:45 GMT
Tyler Hale created MAPREDUCE-6414:
-------------------------------------

             Summary: Distcp command very slow to enumerate files needing
                 Key: MAPREDUCE-6414
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6414
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: distcp
    Affects Versions: 2.5.0
         Environment: RHEL 6.5
            Reporter: Tyler Hale


When copying large amounts of data using distcp utility (100's of TBs), the distcp utility
takes a large time to enumerate all of the files that have changed.  In my system, this corresponds
to 14-16 hours before the actual copying of data begins. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message