hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-7800) Bind node constraint once a container is proposed to be placed on this node
Date Wed, 24 Jan 2018 03:19:01 GMT
Weiwei Yang created YARN-7800:
---------------------------------

             Summary: Bind node constraint once a container is proposed to be placed on this
node
                 Key: YARN-7800
                 URL: https://issues.apache.org/jira/browse/YARN-7800
             Project: Hadoop YARN
          Issue Type: Sub-task
          Components: RM
            Reporter: Weiwei Yang


We found when there is circular dependency between multiple scheduling requests, allocation
decisions made by placement constraint algorithm might be conflicting with related tags. More
describing of the issue please refer to YARN-7783.

To solve this issue, a possible solution is to bind *node constraint*. If the algorithm
wants to place any new container to this node, except for checking if it satisfies the placement
constraint, it also check if it satisfies the node constraint. For example

1) "foo", anti-affinity with "foo"

+Implies node constraint:+ on each node, it cannot have more than 1 foo tags

2) "bar", anti-affinity with "foo"

+Implies node constraint:+ on each node, it cannot have both "bar" and "foo" tags

With such constraint, it works like
 * req2 is placed on any of nodes, e.g n2, +a node constraint[1] is added to n2 that constrains
this node cannot have both "bar" and "foo" tags+
 * when the algorithm wants to place req1 on n2, it checks if its placement constraint is
satisfied. It should be as there is no foo container on this node yet.
 * Then the algorithm checks if the node constraint is satisfied. It is not because it violate
node constraint [1].

This avoids to do additional re-attempt like what was done in YARN-7783.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message