Jim Brennan created YARN-10475:
----------------------------------
Summary: Scale RM-NM heartbeat interval based on node utilization
Key: YARN-10475
URL: https://issues.apache.org/jira/browse/YARN-10475
Project: Hadoop YARN
Issue Type: Improvement
Components: yarn
Affects Versions: 2.10.1, 3.4.1
Reporter: Jim Brennan
Assignee: Jim Brennan
Add the ability to scale the RM-NM heartbeat interval based on node cpu utilization compared
to overall cluster cpu utilization. If a node is over-utilized compared to the rest of the
cluster, it's heartbeat interval slows down. If it is under-utilized compared to the rest
of the cluster, it's heartbeat interval speeds up.
This is a feature we have been running with internally in production for several years.
It was developed by [~nroberts], based on the observation that larger faster nodes on our
cluster were under-utilized compared to smaller slower nodes.
This feature is dependent on [YARN-10450], which added cluster-wide utilization metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
|