hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-678) distcp -m option does not work
Date Wed, 23 Jul 2014 18:19:40 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allen Wittenauer resolved MAPREDUCE-678.
----------------------------------------

    Resolution: Cannot Reproduce

> distcp -m option does not work 
> -------------------------------
>
>                 Key: MAPREDUCE-678
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-678
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: distcp
>    Affects Versions: 0.20.1
>            Reporter: Suhas Gogate
>
> In spite of -m 10 , number of launched map tasks are 15.  
> bin/hadoop  distcp -Dmapred.job.queue.name=xxxx -i -p -update -m 10   -delete  hftp://xxxx.yyyy.com:50070/user/xxxx/QA_incremental
hdfs://aaaa.yyyy.com:8020/user/xxxxx/QA_incremental'
> 09/06/29 19:14:24 INFO tools.DistCp: srcPaths=[hftp://xxxxx.yyyy.com:50070/user/xxxx/QA_incremental]
> 09/06/29 19:14:24 INFO tools.DistCp: destPath=hdfs://zzzz.yyyy.com:8020/user/xxxx/QA_incremental
> 09/06/29 19:14:34 INFO tools.DistCp: sourcePathsCount=62
> 09/06/29 19:14:34 INFO tools.DistCp: filesToCopyCount=15
> 09/06/29 19:14:34 INFO tools.DistCp: bytesToCopyCount=18.0g
> 09/06/29 19:14:37 INFO mapred.JobClient: Running job: job_200906290541_1523
> 09/06/29 19:14:38 INFO mapred.JobClient:  map 0% reduce 0%
> 09/06/29 19:15:22 INFO mapred.JobClient:  map 2% reduce 0%
> 09/06/29 19:26:57 INFO mapred.JobClient:  map 9% reduce 0%
> 09/06/29 19:36:02 INFO mapred.JobClient:  map 16% reduce 0%
> 09/06/29 19:36:46 INFO mapred.JobClient:  map 22% reduce 0%
> 09/06/29 19:37:25 INFO mapred.JobClient:  map 29% reduce 0%
> 09/06/29 19:38:28 INFO mapred.JobClient:  map 36% reduce 0%
> 09/06/29 19:39:41 INFO mapred.JobClient:  map 42% reduce 0%
> 09/06/29 19:40:08 INFO mapred.JobClient:  map 46% reduce 0%
> 09/06/29 19:44:08 INFO mapred.JobClient:  map 53% reduce 0%
> 09/06/29 19:44:19 INFO mapred.JobClient:  map 60% reduce 0%
> 09/06/29 19:44:39 INFO mapred.JobClient:  map 66% reduce 0%
> 09/06/29 19:45:04 INFO mapred.JobClient:  map 73% reduce 0%
> 09/06/29 19:46:08 INFO mapred.JobClient:  map 80% reduce 0%
> 09/06/29 19:46:24 INFO mapred.JobClient:  map 86% reduce 0%
> 09/06/29 19:46:38 INFO mapred.JobClient:  map 93% reduce 0%
> 09/06/29 19:48:00 INFO mapred.JobClient:  map 100% reduce 0%
> 09/06/29 19:48:19 INFO mapred.JobClient: Job complete: job_200906290541_1523
> 09/06/29 19:48:19 INFO mapred.JobClient: Counters: 10
> 09/06/29 19:48:19 INFO mapred.JobClient:   Job Counters 
> 09/06/29 19:48:19 INFO mapred.JobClient:     Launched map tasks=15
> 09/06/29 19:48:19 INFO mapred.JobClient:   FileSystemCounters
> 09/06/29 19:48:19 INFO mapred.JobClient:     HDFS_BYTES_READ=7742
> 09/06/29 19:48:19 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=19294982191
> 09/06/29 19:48:19 INFO mapred.JobClient:   distcp
> 09/06/29 19:48:19 INFO mapred.JobClient:     Files copied=15
> 09/06/29 19:48:19 INFO mapred.JobClient:     Bytes copied=19294982191
> 09/06/29 19:48:19 INFO mapred.JobClient:     Bytes expected=19294982191
> 09/06/29 19:48:19 INFO mapred.JobClient:   Map-Reduce Framework
> 09/06/29 19:48:19 INFO mapred.JobClient:     Map input records=16
> 09/06/29 19:48:19 INFO mapred.JobClient:     Spilled Records=0
> 09/06/29 19:48:19 INFO mapred.JobClient:     Map input bytes=3190
> 09/06/29 19:48:19 INFO mapred.JobClient:     Map output records=0



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message