hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantinos Karanasos <kkarana...@gmail.com>
Subject Re: [DISCUSS] Merge YARN-6592 to trunk
Date Fri, 26 Jan 2018 03:08:46 GMT
Thanks for starting the thread Arun, +1 from me too.

Konstantinos

On Thu, Jan 25, 2018 at 18:54 Weiwei Yang <cheersyang@hotmail.com> wrote:

> +1, thanks for getting to this milestone Arun.
> I’ve done some basic validations on a 4 nodes cluster, with some general
> affinity/anti-affinty/cardinality constraints, it worked. I’ve also
> reviewed the doc, it’s in good shape and very illustrative.
>
> Thanks.
>
> --
> Weiwei
>
> On 26 Jan 2018, 10:44 AM +0800, Sunil G <sunilg@apache.org>, wrote:
> +1.
>
> Thanks Arun.
>
> I did manual testing for check affinity and anti-affinity features with
> placement allocator. Also checked SLS to see any performance regression,
> and there are not much difference as Arun mentioned.
>
> Thanks all the folks for working on this. Kudos!
>
> - Sunil
>
>
> On Fri, Jan 26, 2018 at 5:16 AM Arun Suresh <asuresh@apache.org<mailto:
> asuresh@apache.org>> wrote:
> Hello yarn-dev@
>
> We feel that the YARN-6592 dev branch mostly in shape to be merged into
> trunk. This branch adds support for placing containers in YARN using rich
> placement constraints. For example, this can be used by applications to
> co-locate containers on a node or rack (affinity constraint), spread
> containers across nodes or racks (anti-affinity constraint), or even
> specify the maximum number of containers on a node/rack (cardinality
> constraint).
>
> We have integrated this feature into the Distributed-Shell application for
> feature testing. We have performed end-to-end testing on moderately-sized
> clusters to verify that constraints work fine. Performance tests have been
> done via both SLS tests and Capacity Scheduler performance unit tests, and
> no regression was found. We have opened a JIRA to track Jenkins acceptance
> of the aggregated patch [2]. Documentation is in the process of being
> completed [3]. You can also check our design document for more details [4].
>
> Config flags are needed to enable this feature and it should not have any
> effect on YARN when turned off. Once merged, we plan to work on further
> improvements, which can be tracked in the umbrella YARN-7812 [5].
>
> Kindly do take a look at the branch and raise issues/concerns that need to
> be addressed before the merge.
>
> Many thanks to Konstantinos, Wangda, Panagiotis, Weiwei, and Sunil for
> their contributions to this effort, as well as Subru, Chris, Carlo, and
> Vinod for their inputs and discussions.
>
> Cheers
> -Arun
>
>
> [1] https://issues.apache.org/jira/browse/YARN-6592
> [2] https://issues.apache.org/jira/browse/YARN-7792
> [3] https://issues.apache.org/jira/browse/YARN-7780
> [4]
> https://issues.apache.org/jira/secure/attachment/12867869/YARN-6592-Rich-Placement-Constraints-Design-V1.pdf
> [5] https://issues.apache.org/jira/browse/YARN-7812
>
> --
Konstantinos

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message