hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gour Saha <gs...@hortonworks.com>
Subject Re: [Vote] Merge discussion for Node attribute support feature YARN-3409
Date Fri, 07 Sep 2018 03:48:47 GMT
+1 for merge

-Gour


> On Sep 6, 2018, at 8:44 PM, Sunil G <sunilg@apache.org> wrote:
> 
> +1 for the merge.
> 
> - Sunil
> 
> 
> On Wed, Sep 5, 2018 at 6:01 PM Naganarasimha Garla <
> naganarasimha_gr@apache.org> wrote:
> 
>> Hi All,
>>         Thanks for feedback folks, based on the positive response starting
>> a Vote thread for merging YARN-3409 to master.
>> 
>> Regards,
>> + Naga & Sunil
>> 
>>> On Wed, 5 Sep 2018 2:51 am Wangda Tan, <wheeleast@gmail.com> wrote:
>>> 
>>> +1 for the merge, it gonna be a great addition to 3.2.0 release. Thanks
>> to
>>> everybody for pushing this feature to complete.
>>> 
>>> Best,
>>> Wangda
>>> 
>>> On Tue, Sep 4, 2018 at 8:25 AM Bibinchundatt <bibin.chundatt@huawei.com>
>>> wrote:
>>> 
>>>> +1 for merge. Fetaure would be a good addition to 3.2 release.
>>>> 
>>>> --------------------------------------------------
>>>> Bibin A Chundatt
>>>> M: +91-9742095715 <+91%2097420%2095715><tel:+91-9742095715
>> <+91%2097420%2095715>>
>>>> E: bibin.chundatt@huawei.com<mailto:bibin.chundatt@huawei.com>
>>>> 2012<tel:2012>实验室-印研IT&Cloud BU分部
>>>> 2012<tel:2012> Laboratories-IT&Cloud BU Branch Dept.
>>>> From:Naganarasimha Garla
>>>> To:common-dev@hadoop.apache.org,Hdfs-dev,yarn-dev@hadoop.apache.org,
>>>> mapreduce-dev@hadoop.apache.org,
>>>> Date:2018-08-29 20:00:44
>>>> Subject:[Discuss] Merge discussion for Node attribute support feature
>>>> YARN-3409
>>>> 
>>>> Hi All,
>>>> 
>>>> We would like to hear your thoughts on merging “Node Attributes Support
>> in
>>>> YARN” branch (YARN-3409) [2] into trunk in a few weeks. The goal is to
>> get
>>>> it in for HADOOP 3.2.
>>>> 
>>>> *Major work happened in this branch*
>>>> 
>>>> YARN-6858. Attribute Manager to store and provide node attributes in RM
>>>> YARN-7871. Support Node attributes reporting from NM to RM( distributed
>>>> node attributes)
>>>> YARN-7863. Modify placement constraints to support node attributes
>>>> YARN-7875. Node Attribute store for storing and recovering attributes
>>>> 
>>>> *Detailed Design:*
>>>> 
>>>> Please refer [1] for detailed design document.
>>>> 
>>>> *Testing Efforts:*
>>>> 
>>>> We did detailed tests for the feature in the last few weeks.
>>>> This feature will be enabled only when Node Attributes constraints are
>>>> specified through SchedulingRequest from AM.
>>>> Manager implementation will help to store and recover Node Attributes.
>>>> This
>>>> works with existing placement constraints.
>>>> 
>>>> *Regarding to API stability:*
>>>> 
>>>> All newly added @Public APIs are @Unstable.
>>>> 
>>>> Documentation jira [3] could help to provide detailed configuration
>>>> details. This feature works from end-to-end and we tested this in our
>>>> local
>>>> cluster. Branch code is run against trunk and tracked via [4].
>>>> 
>>>> We would love to get your thoughts before opening a voting thread.
>>>> 
>>>> Special thanks to a team of folks who worked hard and contributed
>> towards
>>>> this efforts including design discussion / patch / reviews, etc.: Weiwei
>>>> Yang, Bibin Chundatt, Wangda Tan, Vinod Kumar Vavilappali, Konstantinos
>>>> Karanasos, Arun Suresh, Varun Saxena, Devaraj Kavali, Lei Guo, Chong
>> Chen.
>>>> 
>>>> [1] :
>>>> 
>>>> 
>> https://issues.apache.org/jira/secure/attachment/12937633/Node-Attributes-Requirements-Design-doc_v2.pdf
>>>> [2] : https://issues.apache.org/jira/browse/YARN-3409
>>>> [3] : https://issues.apache.org/jira/browse/YARN-7865
>>>> [4] : https://issues.apache.org/jira/browse/YARN-8718
>>>> 
>>>> Thanks,
>>>> + Naga & Sunil Govindan
>>>> 
>>> 
>> 
Mime
View raw message