hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sreejith Ramakrishnan <sreejith.c...@gmail.com>
Subject Re: My Job always in 'SUBMIT' state. Stuck at Map: 0% - Reduce 0%
Date Thu, 11 Jul 2013 07:55:42 GMT
@Devaraj K

*I tried explicitly calling incomingJob.initTasks()*and it gave me a
pleasant INFO org.apache.hadoop.mapred.JobInProgress: Job
job_201307111248_0002 initialized successfully with 1 map tasks and 1
reduce tasks

*Yet the **Cannot create task split for <my-job-id>** is still there*.
About errors relating to the JobID, is this significant? --- INFO
org.apache.hadoop.mapred.JobInProgress: job_201307111248_0002: nMaps=1
nReduces=1 max=-1

*P.S: In the log file I shared, anything with "sreejith == " were logged by
me. Not system-generated*


On Thu, Jul 11, 2013 at 1:18 PM, Sreejith Ramakrishnan <
sreejith.code@gmail.com> wrote:

> I've attached the relevant portion of the log.
>
> For easy testing, *I've attached the maven project* for the
> EDFJobScheduler. Just make a JAR from it and put it in $HADOOP_HOME/lib and
> add this property to *mapred-site.xml*:
>
> *<property>
> *
> *    <name>mapred.jobtracker.taskScheduler</name>
>     <value>org.apache.hadoop.mapred.EDFJobScheduler</value>
> *
> *</property>*
>
>
> On Thu, Jul 11, 2013 at 12:19 PM, Sreejith Ramakrishnan <
> sreejith.code@gmail.com> wrote:
>
>> I'm making my own scheduler. Since I had errors, I rewrote the scheduler
>> to do just a simple thing. When I give it a job, it allocates 1 map() and 1
>> reduce() to it. But that isn't happening. I used a barebones WordCount pgm.
>> I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:
>>
>> INFO org.apache.hadoop.mapred.JobInProgress: Cannot create task split for
>> <my-job-id>
>>
>> DETAILED INFO:
>>
>> Here's some snippets of code. The scheduler logic is inside the
>> assignTasks():
>>
>> We get a JobInProgressListener in the constructor-
>> this.jobQueueJobInProgressListener = new JobQueueJobInProgressListener();
>>
>> In start() -
>>
>> this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgressListener);
>>
>> Then I just take the last element in that queue (I assume its the latest
>> submitted job) and assign to incomingJob (of type JobInProgress).
>>
>> Then I obtain a new map task as follows and append this task to a list
>> which is the return value of the assignTasks():
>> Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus,
>> numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts());
>>
>>
>> I've attached the scheduler. Can any expert point out if there's any
>> ignorant mistakes from my side?
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message