hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh Kumar <sureshapache...@gmail.com>
Subject Re: Shuffling over the network for local map data.
Date Tue, 22 Jan 2013 17:36:28 GMT
Hi Steve,

My assumption is that unless it is reading from  http://127.0.0.1/ or
http://localhost/ , it reads over the network. If I'm wrong please correct
me. The http tracker address that a ReduceTask receives is not of that
format. So I do not think it is reading using the loop back address.

Thanks,
Suresh.



On Tue, Jan 22, 2013 at 8:46 AM, Steve Loughran <stevel@hortonworks.com>wrote:

> It's just using the loopback address, right -not going on to the external
> network and back again?
>
> On 22 January 2013 03:22, Suresh Kumar <sureshapachedev@gmail.com> wrote:
>
> > Hello,
> >
> > I noticed that the shuffle phase is reading data over http even when data
> > is available locally. The version of hadoop I'm using is 1.0.3. Is there
> a
> > reason it is implemented this way ? Is it OK to make a change that will
> > identify that the data is available locally and read from the local disk
> > instead of the http?
> >
> > I'm new to this developer list and apache developer list in general. So
> > please feel free to let me know if there is a certain etiquette that I'm
> > not following.
> >
> > Thanks,
> > Suresh.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message