hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tsuyoshi OZAWA <ozawa.tsuyo...@gmail.com>
Subject Re: The code for the Shuffle phase of MapReduce
Date Fri, 14 Feb 2014 00:29:51 GMT
Hi Pramod,

In addition to shuffle plugin related parts, please check out how
spill works in MapTask.java and how reducer fetches MapOutputs from
mappers.

Thanks, Tsuyoshi

On Mon, Feb 10, 2014 at 2:54 AM, Pramod Biligiri
<pramodbiligiri@gmail.com> wrote:
> Hi,
> I'm beginning to look at the code for the Shuffle phase of MapReduce, for
> an academic project.
>
> I wanted to confirm if I have started at the right location:
> I'm looking at a file called ShuffleHandler.java in the
> hadoop-mapreduce-client-shuffle sub project of hadoop-mapreduce-project.
>
> Are there other packages/projects I should be examining that are closely
> related to this?
>
> Thanks,
> Pramod



-- 
- Tsuyoshi

Mime
View raw message