hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5786) Support Keep-Alive connections in ShuffleHandler
Date Sun, 09 Mar 2014 00:20:42 GMT
Rajesh Balamohan created MAPREDUCE-5786:

             Summary: Support Keep-Alive connections in ShuffleHandler
                 Key: MAPREDUCE-5786
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5786
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 2.4.0
            Reporter: Rajesh Balamohan
            Assignee: Rajesh Balamohan

Currently ShuffleHandler supports fetching map-outputs in batches from same host.  But there
are scenarios wherein, fetchers pull data aggressively (i.e start pulling the data as &
when they are available).  In this case, the number of mapIds that are pulled from same host
remains at 1. This causes lots of connections to be established.

Number of connections can be reduced a lot if ShuffleHandler supports Keep-Alive.

This message was sent by Atlassian JIRA

View raw message