hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prabhu Joseph (Jira)" <j...@apache.org>
Subject [jira] [Created] (YARN-10139) ValidateAndGetSchedulerConfiguration API fails when cluster max allocation > default 8GB
Date Fri, 14 Feb 2020 15:50:00 GMT
Prabhu Joseph created YARN-10139:

             Summary: ValidateAndGetSchedulerConfiguration API fails when cluster max allocation
> default 8GB
                 Key: YARN-10139
                 URL: https://issues.apache.org/jira/browse/YARN-10139
             Project: Hadoop YARN
          Issue Type: Bug
    Affects Versions: 3.3.0
            Reporter: Prabhu Joseph
            Assignee: Prabhu Joseph

ValidateAndGetSchedulerConfiguration fails when the cluster max allocation (yarn.scheduler.maximum-allocation-mb)
is set to resource (eg: 16GB) > default 8GB in yarn-site.xml.

As part of validation API, there are two configuration used - CapacitySchedulerConfiguration
and Configuration (yarn-site.xml). When CapacityScheduler is initialized with CapacitySchedulerConfiguration,
as part of queues initialization, it checks the queue maximum allocation which is not present
and so checks cluster max allocation which is not present (it is present only in YarnConfiguration)
and defaults to 8GB. This will fail as queue max allocation 8GB is decreased from previous

2020-02-14 07:38:46,087 WARN org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices:
CapacityScheduler configuration validation failed:java.io.IOException: Failed to re-init queues
: Trying to reinitialize root.default.c1.c3 the maximum allocation size can not be decreased!
Current setting: <memory:164860, vCores:88>, trying to set it to: <memory:8192, vCores:4>

CapacityScheduler initialize code reads a yarn config from CapacitySchedulerConfiguration
causing the issue. 

This message was sent by Atlassian Jira

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message