hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-4954) ShuffleHandler logs excessively when channel closed prematurely by reducer
Date Tue, 22 Jan 2013 16:24:12 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-4954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jason Lowe resolved MAPREDUCE-4954.

    Resolution: Not A Problem

Sorry for the noise, I discovered the nodemanager in question was running 0.23.4 which did
not have the fix for MAPREDUCE-4801.
> ShuffleHandler logs excessively when channel closed prematurely by reducer
> --------------------------------------------------------------------------
>                 Key: MAPREDUCE-4954
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4954
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.0.3-alpha, 0.23.5
>            Reporter: Jason Lowe
> As initially reported in MAPREDUCE-4801, the shuffle protocol allows a reducer to hangup
the connection to a nodemanager after receiving the shuffle header.  This can introduce a
connection reset IOException or a ClosedChannelException on the nodemanager.  Logging these
is not very interesting, and adds a lot of data to the NM log.
> MAPREDUCE-4801 missed some places where these exceptions can occur, will post sample
exceptions below for reference.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message