hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naganarasimha Garla <naganarasimha...@apache.org>
Subject [Discuss] Merge discussion for Node attribute support feature YARN-3409
Date Wed, 29 Aug 2018 14:30:19 GMT
Hi All,

We would like to hear your thoughts on merging “Node Attributes Support in
YARN” branch (YARN-3409) [2] into trunk in a few weeks. The goal is to get
it in for HADOOP 3.2.

*Major work happened in this branch*

YARN-6858. Attribute Manager to store and provide node attributes in RM
YARN-7871. Support Node attributes reporting from NM to RM( distributed
node attributes)
YARN-7863. Modify placement constraints to support node attributes
YARN-7875. Node Attribute store for storing and recovering attributes

*Detailed Design:*

Please refer [1] for detailed design document.

*Testing Efforts:*

We did detailed tests for the feature in the last few weeks.
This feature will be enabled only when Node Attributes constraints are
specified through SchedulingRequest from AM.
Manager implementation will help to store and recover Node Attributes. This
works with existing placement constraints.

*Regarding to API stability:*

All newly added @Public APIs are @Unstable.

Documentation jira [3] could help to provide detailed configuration
details. This feature works from end-to-end and we tested this in our local
cluster. Branch code is run against trunk and tracked via [4].

We would love to get your thoughts before opening a voting thread.

Special thanks to a team of folks who worked hard and contributed towards
this efforts including design discussion / patch / reviews, etc.: Weiwei
Yang, Bibin Chundatt, Wangda Tan, Vinod Kumar Vavilappali, Konstantinos
Karanasos, Arun Suresh, Varun Saxena, Devaraj Kavali, Lei Guo, Chong Chen.

[1] :
[2] : https://issues.apache.org/jira/browse/YARN-3409
[3] : https://issues.apache.org/jira/browse/YARN-7865
[4] : https://issues.apache.org/jira/browse/YARN-8718

+ Naga & Sunil Govindan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message