hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wangda Tan <wheele...@gmail.com>
Subject Re: [Discuss] Merge discussion for Node attribute support feature YARN-3409
Date Tue, 04 Sep 2018 18:50:45 GMT
+1 for the merge, it gonna be a great addition to 3.2.0 release. Thanks to
everybody for pushing this feature to complete.

Best,
Wangda

On Tue, Sep 4, 2018 at 8:25 AM Bibinchundatt <bibin.chundatt@huawei.com>
wrote:

> +1 for merge. Fetaure would be a good addition to 3.2 release.
>
> --------------------------------------------------
> Bibin A Chundatt
> M: +91-9742095715<tel:+91-9742095715>
> E: bibin.chundatt@huawei.com<mailto:bibin.chundatt@huawei.com>
> 2012<tel:2012>实验室-印研IT&Cloud BU分部
> 2012<tel:2012> Laboratories-IT&Cloud BU Branch Dept.
> From:Naganarasimha Garla
> To:common-dev@hadoop.apache.org,Hdfs-dev,yarn-dev@hadoop.apache.org,
> mapreduce-dev@hadoop.apache.org,
> Date:2018-08-29 20:00:44
> Subject:[Discuss] Merge discussion for Node attribute support feature
> YARN-3409
>
> Hi All,
>
> We would like to hear your thoughts on merging “Node Attributes Support in
> YARN” branch (YARN-3409) [2] into trunk in a few weeks. The goal is to get
> it in for HADOOP 3.2.
>
> *Major work happened in this branch*
>
> YARN-6858. Attribute Manager to store and provide node attributes in RM
> YARN-7871. Support Node attributes reporting from NM to RM( distributed
> node attributes)
> YARN-7863. Modify placement constraints to support node attributes
> YARN-7875. Node Attribute store for storing and recovering attributes
>
> *Detailed Design:*
>
> Please refer [1] for detailed design document.
>
> *Testing Efforts:*
>
> We did detailed tests for the feature in the last few weeks.
> This feature will be enabled only when Node Attributes constraints are
> specified through SchedulingRequest from AM.
> Manager implementation will help to store and recover Node Attributes. This
> works with existing placement constraints.
>
> *Regarding to API stability:*
>
> All newly added @Public APIs are @Unstable.
>
> Documentation jira [3] could help to provide detailed configuration
> details. This feature works from end-to-end and we tested this in our local
> cluster. Branch code is run against trunk and tracked via [4].
>
> We would love to get your thoughts before opening a voting thread.
>
> Special thanks to a team of folks who worked hard and contributed towards
> this efforts including design discussion / patch / reviews, etc.: Weiwei
> Yang, Bibin Chundatt, Wangda Tan, Vinod Kumar Vavilappali, Konstantinos
> Karanasos, Arun Suresh, Varun Saxena, Devaraj Kavali, Lei Guo, Chong Chen.
>
> [1] :
>
> https://issues.apache.org/jira/secure/attachment/12937633/Node-Attributes-Requirements-Design-doc_v2.pdf
> [2] : https://issues.apache.org/jira/browse/YARN-3409
> [3] : https://issues.apache.org/jira/browse/YARN-7865
> [4] : https://issues.apache.org/jira/browse/YARN-8718
>
> Thanks,
> + Naga & Sunil Govindan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message