hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thanh Do (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-5479) NameNode should not send empty block replication request to DataNode
Date Sat, 07 Jan 2012 14:12:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181970#comment-13181970

Thanh Do commented on HADOOP-5479:

Hi Hairong,

This is an old issue but could you please verify my understanding about this bug? So basically,
because the NN keeps send empty replication command, the DN creates a new thread for every
empty command, which eats up DN's resource, right?

Many thanks
> NameNode should not send empty block replication request to DataNode
> --------------------------------------------------------------------
>                 Key: HADOOP-5479
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5479
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>            Priority: Critical
>             Fix For: 0.19.2, 0.20.0
>         Attachments: numTransfers.patch, numTransfers1.patch
> On our production clusters, we occasionally see that NameNode sends an empty block replication
request to DataNode on  every heartbeat, thus blocking this DataNode from replicating or deleting
any block.
> This is partly caused by DataNode sending a wrong number of replications in progress
which will be fixed by HADOOP-5465. There is also a flaw at the NameNode side. NameNode should
not interpret the number of replications in progress as the number of targets since replication
is done through a pipeline. It also should make sure that no empty replication request is
sent to DataNode.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message