hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11676) Add API to NetworkTopology for getting all racks
Date Fri, 13 Mar 2015 03:21:38 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359874#comment-14359874
] 

Walter Su commented on HADOOP-11676:
------------------------------------

Thanks [~zhz] for thorough review. Good point using ArrayList instead of a HashSet. Also because
racks.add is more often, and racks.remove is less often, there is no performance issue using
ArrayList. And sorry for lack of code comments.
Thanks [~aw], we should deal with HDFS expansion very carefully. Thanks [~zhz] again for answering
the question.

> Add API to NetworkTopology for getting all racks
> ------------------------------------------------
>
>                 Key: HADOOP-11676
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11676
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HADOOP-11676.patch
>
>
> The existing two NetworkTopology.chooseRandom(..) API support choosing node from scope
and choosing outside scope. BlockPlacementPolicyDefault class use these two API to choose
node from one rack or choose outside one rack.
> We want to implement a new placement policy called BlockPlacementPolicyFaultTolerant
which tries its best to place replicas to most racks. To achieve this, We need to know how
many replicas each rack has. And first, we need to get all racks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message