hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhinaba Sarkar (Jira)" <j...@apache.org>
Subject [jira] [Created] (YARN-10661) Add option to scale down specific containers in flex API
Date Tue, 02 Mar 2021 04:55:00 GMT
Abhinaba Sarkar created YARN-10661:

             Summary: Add option to scale down specific containers in flex API
                 Key: YARN-10661
                 URL: https://issues.apache.org/jira/browse/YARN-10661
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: yarn
            Reporter: Abhinaba Sarkar

We can flex the number of containers in yarn-service. But currently we do not have option
to specify which containers are to be scaled down. This is useful in scaling down Hive LLAP
clusters. For example - 
2021-02-22 12:52:58,983 [IPC Server handler 0 on 24799] INFO  service.ClientAMService - Flexing component llap to 42021-02-22 12:52:58,983 [Component  dispatcher] INFO  component.Component - [FLEX DOWN COMPONENT llap]: scaling down from 6 to 42021-02-22 12:52:58,984 [Component  dispatcher] INFO  instance.ComponentInstance - [COMPINSTANCE llap-5 : container_e02_1613896771789_0071_01_000007]: Flexed down by user, destroying.2021-02-22 12:52:58,987 [Component  dispatcher] INFO  instance.ComponentInstance - [COMPINSTANCE llap-4 : container_e02_1613896771789_0071_01_000006]: Flexed down by user, destroying.

Last added containers are first ones to be destroyed. This might not be the most optimal approach
as we would like the least loaded(in terms of executors) containers to be destroyed first.
We propose an option to add the list of containers that can be scaled down by flexing. 

This message was sent by Atlassian Jira

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message