hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 최종원 <jongwons.c...@gmail.com>
Subject My Pig script always assign one reduce job in HADOOP 2.2.0 Yarn Resource manager. What was wrong?
Date Wed, 29 Jan 2014 06:53:15 GMT
Hello, I'm new one to this mailing list.

I have been used HADOOP 1.2.1 server, and execute many pig jobs there.

And recently, I considered to change my Hadoop server to HADOOP 2.2.0.

So I tried some pig jobs in HADOOP 2.2.0, as I did in HADOOP 1.2.1 version.

But one thing I hardly understand in YARN MR2, is that Only ONE reduce job
scheduled in every mr job.

At first time, I think that ok, reduce is faster than mr1, because Resource
manager efficiently scheduled reduce job by handling it in only one server.

But in every big size mr job, YARN MR2 allocate Only ONE Reduce job
scheduled every time.

Below is the Extream case.


My Old HADOOP(version 1.2.1) server is consist of 1 jobtracker and 2
tasktracker. (each 4-core, 32G)

KindTotal Tasks(successful+failed+killed)Successful tasksFailed tasksKilled
tasksStart TimeFinish
TimeSetup1<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=SETUP&status=all>
1<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=SETUP&status=SUCCESS>
0<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=SETUP&status=FAILED>
0<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=SETUP&status=KILLED>27-1월-2014
18:01:4527-Jan-2014 18:01:46
(0sec)Map2425<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=MAP&status=all>
2423<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=MAP&status=SUCCESS>
0<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=MAP&status=FAILED>
2<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=MAP&status=KILLED>27-1월-2014
18:01:2627-Jan-2014 19:08:58 (1hrs, 7mins,
31sec)Reduce166<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=REDUCE&status=all>
163<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=REDUCE&status=SUCCESS>
0<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=REDUCE&status=FAILED>
3<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=REDUCE&status=KILLED>27-1월-2014
18:04:3527-Jan-2014 20:40:15 (2hrs, 35mins,
40sec)Cleanup1<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=CLEANUP&status=all>
1<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=CLEANUP&status=SUCCESS>
0<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=CLEANUP&status=FAILED>
0<http://1.234.4.55:50030/jobtaskshistory.jsp?logFile=file:/home/hadoop/hadoop-1.2.0/logs/history/done/version-1/secondnode_1389624460639_/2014/01/27/000012/job_201401132347_12834_1390813304598_root_PigLatin%253ADefaultJobName&taskType=CLEANUP&status=KILLED>27-1월-2014
20:40:1627-Jan-2014 20:40:17 (1sec)

It takes 2 hour and 38 minute.


My New HADOOP(version 2.2.0) server is consist of 1 Resource Manager and 8
Node manager.(each 4-core, 32G) (New system is much better)

Job Name:PigLatin:DefaultJobNameUser Name:hduserQueue:defaultState:SUCCEEDED
Uberized:falseStarted:Tue Jan 28 16:09:41 KST 2014Finished:Tue Jan 28
21:47:45 KST 2014Elapsed:5hrs, 38mins, 4secDiagnostics:Average Map
Time41secAverage
Reduce Time3hrs, 48mins, 23secAverage Shuffle Time1hrs, 36mins, 35secAverage
Merge Time1hrs, 27mins, 38sec
ApplicationMasterAttempt NumberStart TimeNodeLogs1Tue Jan 28 16:09:39 KST
2014awdatanode2:8042logs<http://pvnamenode2:19888/jobhistory/logs/awdatanode2:42860/container_1390892908969_0001_01_000001/job_1390892908969_0001/hduser>
Task TypeTotalCompleteMap<http://pvnamenode2:19888/jobhistory/tasks/job_1390892908969_0001/m>
11721172Reduce<http://pvnamenode2:19888/jobhistory/tasks/job_1390892908969_0001/r>
11Attempt TypeFailedKilledSuccessfulMaps0<http://pvnamenode2:19888/jobhistory/attempts/job_1390892908969_0001/m/FAILED>
1<http://pvnamenode2:19888/jobhistory/attempts/job_1390892908969_0001/m/KILLED>
1172<http://pvnamenode2:19888/jobhistory/attempts/job_1390892908969_0001/m/SUCCESSFUL>
Reduces0<http://pvnamenode2:19888/jobhistory/attempts/job_1390892908969_0001/r/FAILED>
0<http://pvnamenode2:19888/jobhistory/attempts/job_1390892908969_0001/r/KILLED>
1<http://pvnamenode2:19888/jobhistory/attempts/job_1390892908969_0001/r/SUCCESSFUL>
It takes 5 hour and 38 minutes.


Although My Old Hadoop server has poor resouce, It's much faster than New
Hadoop. because reduce jobs distributed.
On the other end, HADOOP 2.2.0 server has rich resources, and, map was much
faster than old system, but the reduce takes terribly long time.

Hadoop 2.2 memory configured as Map (4G, heap space 3G) and Reduce (8G,
heap space 6G).
and I tried various configurations set. but result was always one reduce
job.

What did I wrong ?
How can I distribute the reduce job as It does in HADOOP 1.x ?
Any advices will be greate help for me.

thank you.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message