hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tina Samuel <tinasamue...@gmail.com>
Subject Re: Duplicating map reduce tasks
Date Wed, 02 Jul 2014 13:44:19 GMT
Hi,

Thanks for responding.Is it possible to achieve my requirement by modifying
some code of map reduce,by which I will be able to replicate every map
reduce task that is submitted?

Thanks & Regards,
Tina


Regards,
Tina


On Wed, Jul 2, 2014 at 6:47 PM, Wellington Chevreuil <
wellington.chevreuil@gmail.com> wrote:

> Hi Tina,
>
> That's not controllable with hadoop mr. Hadoop sometimes will do this on
> its own (if you have speculative execution enabled), just for performance
> reasons case a given task is taking to much to complete under a given node,
> and running the same code under replica of the same file will always
> produce same results, then it launches the same task under different nodes,
> and once the first is finished, the result is ready and the others are
> killed.
>
> Cheers.
>
> On 2 Jul 2014, at 07:00, Tina Samuel <tinasamuel89@gmail.com> wrote:
>
> > Hi,
> > I would like to provide the map reduce jobs in the following format :-
> >
> > <Map Reduce task> <number_of_replicas>
> >
> > I want to execute the specified task the specified number of
> > times(number_of_replicas) on different nodes and then I want to compare
> the
> > results produced by these task copies. Is it possible to do this in
> Hadoop
> > Map reduce? If not possible, is there any means by which I can modify the
> > code of map reduce so that I can do it?
> >
> > Thanks & Regards,
> > Tina
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message