hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tina Samuel <tinasamue...@gmail.com>
Subject Re: Duplicating map reduce tasks
Date Thu, 03 Jul 2014 07:53:05 GMT
Hi,
Thanks for responding. Is there any method to schedule a map reduce task ,
say WordCount on all the live worker nodes?
Regards,
Tina


Regards,
Tina


On Wed, Jul 2, 2014 at 7:47 PM, sudhakara st <sudhakara.st@gmail.com> wrote:

> I don't think we can achieve this by changing one class or two class, Look
> into from JobSubmitter.java, TaskAttemptID,java ... to speculative
> execution may it get helps you.  Running the code on different replicas of
> the same block will produce will always produce same results,
>
>
> On Wed, Jul 2, 2014 at 7:14 PM, Tina Samuel <tinasamuel89@gmail.com>
> wrote:
>
> > Hi,
> >
> > Thanks for responding.Is it possible to achieve my requirement by
> modifying
> > some code of map reduce,by which I will be able to replicate every map
> > reduce task that is submitted?
> >
> > Thanks & Regards,
> > Tina
> >
> >
> > Regards,
> > Tina
> >
> >
> > On Wed, Jul 2, 2014 at 6:47 PM, Wellington Chevreuil <
> > wellington.chevreuil@gmail.com> wrote:
> >
> > > Hi Tina,
> > >
> > > That's not controllable with hadoop mr. Hadoop sometimes will do this
> on
> > > its own (if you have speculative execution enabled), just for
> performance
> > > reasons case a given task is taking to much to complete under a given
> > node,
> > > and running the same code under replica of the same file will always
> > > produce same results, then it launches the same task under different
> > nodes,
> > > and once the first is finished, the result is ready and the others are
> > > killed.
> > >
> > > Cheers.
> > >
> > > On 2 Jul 2014, at 07:00, Tina Samuel <tinasamuel89@gmail.com> wrote:
> > >
> > > > Hi,
> > > > I would like to provide the map reduce jobs in the following format
> :-
> > > >
> > > > <Map Reduce task> <number_of_replicas>
> > > >
> > > > I want to execute the specified task the specified number of
> > > > times(number_of_replicas) on different nodes and then I want to
> compare
> > > the
> > > > results produced by these task copies. Is it possible to do this in
> > > Hadoop
> > > > Map reduce? If not possible, is there any means by which I can modify
> > the
> > > > code of map reduce so that I can do it?
> > > >
> > > > Thanks & Regards,
> > > > Tina
> > >
> > >
> >
>
>
>
> --
>
> Regards,
> ...sudhakara
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message