hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szilard Nemeth (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-9359) Avoid code duplication in Resources for calculation methods
Date Thu, 07 Mar 2019 15:19:00 GMT
Szilard Nemeth created YARN-9359:

             Summary: Avoid code duplication in Resources for calculation methods
                 Key: YARN-9359
                 URL: https://issues.apache.org/jira/browse/YARN-9359
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Szilard Nemeth

This is a follow-up for YARN-9318, dealing with code duplication issueas, as discussed with
[~templedf] earlier.

Resources has many very similar calculation methods like addTo, subtractFrom, multiply, etc.
These are having extractable code as common, the only difference could be the calculation
they perform on the passed Resource object(s).

These methods either receive one or two Resource objects and make some calculations on these.
One caveat that needs some attention is that some of them do clone the Resource and do the
calculation on the cloned resource and return the result (leaving the passed Resource alone)
and some of them perform the calculation on the passed Resource object itself.

The common code could be extracted like this: 

private static Resource applyFunctionOnValues(Resource lhs,
      Function<Long, Long> valueFunction) {
    int numResources = ResourceUtils.getNumberOfCountableResourceTypes();
    for (int i = 0; i < numResources; i++) {
      try {
        ResourceInformation lhsValue = lhs.getResourceInformation(i);
        Long modifiedValue = valueFunction.apply(lhsValue.getValue());
        lhs.setResourceValue(i, modifiedValue);
      } catch (ResourceNotFoundException ye) {
        LOG.warn("Resource is missing:" + ye.getMessage());
    return lhs;

And an example code could be like this: 

public static Resource multiplyAndRoundUp(Resource lhs, double by) {
    return applyFunctionOnValues(clone(lhs),
        (value) -> (long) Math.ceil(value * by));

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message