hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "He Tianyi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-5510) Optimize Resource implementation for instantiation
Date Thu, 11 Aug 2016 05:56:22 GMT
He Tianyi created YARN-5510:

             Summary: Optimize Resource implementation for instantiation
                 Key: YARN-5510
                 URL: https://issues.apache.org/jira/browse/YARN-5510
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: resourcemanager
    Affects Versions: 2.6.0
            Reporter: He Tianyi

According to YARN-5479, many {{Resource}} instances are created as intermediate result of
calculating in scheduler (as a result of {{Resources.subtract}} and {{Resources.add}}). This
introduces pressure on garbage collector as well as CPU cycles for scheduling.

Therefore, the issue proposes a lightweight implementation of {{Resource}}, which optimized
for scheduler usage. That is, making instantiation of {{Resource}} cheaper.
In discussion of {{YARN-5479}}, we thought maybe sticking to one version of implementation
avoids unnecessary complexity.

In current implementation {{ResourceImplPB}}, a builder is also created during construction.
Also, {{maybeInitBuilder}} get called during each {{setMemory}} or {{setVirtualCores}}. The
{{builder}} get prepared regardless whether {{Resource}} will be exchanged via IPC or not.

However, in scheduler (which performance is critical), most of these instances are just intermediate
result and not exposed.

Suppose we could move builder related code into {{getProto}}.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message