hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8091) Add a topology mapper that reads hostname to rack mappings from a Java properties file
Date Sun, 19 Feb 2012 17:38:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211437#comment-13211437

Aaron T. Myers commented on HADOOP-8091:

Seems like this might be (very nearly) a duplicate of HADOOP-7030. Do you agree, Steve?
> Add a topology mapper that reads hostname to rack mappings from a Java properties file
> --------------------------------------------------------------------------------------
>                 Key: HADOOP-8091
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8091
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: util
>    Affects Versions: 0.24.0
>            Reporter: Steve Loughran
>            Priority: Minor
> Getting mapping scripts right is surprisingly hard -and if its wrong in production bad
things happen. It would be good to have something simpler for beginners -and one that is trivial
to generate by a machine based on infrastructure data.
> I propose adding an alternative mapper, one driven by a java property file
>  # the specific topology mapper must be identified for loading
>  # it uses another key to identify the property file to load. This is checked for on
startup -if missing, fail.
>  # one property, perhaps "default-rack" identifies the default rack mapping for any host
not in the list
>  # every other entry lists a hostname to rack mapping
>  # hostname mapping is done on the first entry in the FQDN, to be less brittle to domain
> Example
> {code}
> default-rack=/rack1
> host1=/rack1
> host2=/rack1
> host3=/rack2
> host4=/rack2
> {code}
> Implementation
>  * add a new mapper that builds a concurrent hash map
>  * read in every entry in the specified property file, add it to the map
>  * when queried, extract the hostname (i.e. everything before any ".")
>  * match that in the hash table, return if found
>  * if not found: return the default rack
> Feature creep would be to poll this file for changes at a (specified) frequency, and
pick up the changes when they occur. This would require removing the caching topology mapper
that wraps all others in the NN and RM.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message