hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thanh Do (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-5479) NameNode should not send empty block replication request to DataNode
Date Sat, 07 Jan 2012 14:12:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181970#comment-13181970
] 

Thanh Do commented on HADOOP-5479:
----------------------------------

Hi Hairong,

This is an old issue but could you please verify my understanding about this bug? So basically,
because the NN keeps send empty replication command, the DN creates a new thread for every
empty command, which eats up DN's resource, right?

Many thanks
                
> NameNode should not send empty block replication request to DataNode
> --------------------------------------------------------------------
>
>                 Key: HADOOP-5479
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5479
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>            Priority: Critical
>             Fix For: 0.19.2, 0.20.0
>
>         Attachments: numTransfers.patch, numTransfers1.patch
>
>
> On our production clusters, we occasionally see that NameNode sends an empty block replication
request to DataNode on  every heartbeat, thus blocking this DataNode from replicating or deleting
any block.
> This is partly caused by DataNode sending a wrong number of replications in progress
which will be fixed by HADOOP-5465. There is also a flaw at the NameNode side. NameNode should
not interpret the number of replications in progress as the number of targets since replication
is done through a pipeline. It also should make sure that no empty replication request is
sent to DataNode.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message