hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11676) Add API to NetworkTopology for getting all racks
Date Thu, 12 Mar 2015 18:09:39 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359098#comment-14359098

Zhe Zhang commented on HADOOP-11676:

Thanks [~aw] for the question. {{getNumOfRacks}} is already getting the number of racks and
this JIRA just extends the logic to return actual list of racks. In other words it won't change
the semantics in regard of staleness. 

That said, IIUC, {{NetworkTopology#add}} should take care of this expansion scenario. 

[~walter.k.su] Thanks for the work! The patch basically looks good to me. A few minor points:
# {{racks}} could be an ArrayList instead of a HashSet. When {{racks.add}} is called, you
already know it's a new rack.
# {{getRacks}} could use some Javadoc
# It took me a while to realize DN0 and DN1 are on the same rack in {{testRacks}}. Some comments
would be helpful.

+1 pending the above minor changes.

It also seems to me this JIRA should go into trunk though. In that case my +1 is non-binding.

> Add API to NetworkTopology for getting all racks
> ------------------------------------------------
>                 Key: HADOOP-11676
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11676
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HADOOP-11676.patch
> The existing two NetworkTopology.chooseRandom(..) API support choosing node from scope
and choosing outside scope. BlockPlacementPolicyDefault class use these two API to choose
node from one rack or choose outside one rack.
> We want to implement a new placement policy called BlockPlacementPolicyFaultTolerant
which tries its best to place replicas to most racks. To achieve this, We need to know how
many replicas each rack has. And first, we need to get all racks.

This message was sent by Atlassian JIRA

View raw message