mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Sekretenko <asekrete...@mesosphere.io>
Subject Re: Review Request 72509: Got rid of tracking reserved offered/allocated in roles tree.
Date Thu, 14 May 2020 14:42:57 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72509/
-----------------------------------------------------------

(Updated May 14, 2020, 2:42 p.m.)


Review request for mesos and Benjamin Mahler.


Bugs: MESOS-10128
    https://issues.apache.org/jira/browse/MESOS-10128


Repository: mesos


Description
-------

Got rid of tracking reserved offered/allocated in roles tree.


Diffs
-----

  src/master/allocator/mesos/hierarchical.hpp 6454cdaa19f776365df34ecf83114f0d6fa20f27 
  src/master/allocator/mesos/hierarchical.cpp 5fe9ffcb518b8427d663ddae43e550795d290e3c 


Diff: https://reviews.apache.org/r/72509/diff/1/


Testing (updated)
-------

`make check`

Ran hierarchical allocator benchmarks, compared with current results on master and 1.9.0 release.
Below are results that improved significantly (compared to 1.10.x branch root) after this
fix.

After fix:
```
[ RUN      ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
Added 2000 agents in 28.968777ms
Added 2000 frameworks in 3.254205437secs
Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter 
Made 2000 allocations in 190.023349ms
Made 0 allocation in 81.404659ms
[       OK ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
(3972 ms)

[ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
Using 10000 agents and 1 frameworks with resource sharedness Regular
Added 1 frameworks in 265991ns
Added 10000 agents in 1.088327374secs
round 0 allocate() took 398.34303ms to make 10000 offers
round 1 allocate() took 436.728365ms to make 10000 offers
round 2 allocate() took 431.187768ms to make 10000 offers
round 3 allocate() took 461.27924ms to make 10000 offers
round 4 allocate() took 426.078997ms to make 10000 offers
round 5 allocate() took 478.756617ms to make 10000 offers
[       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48 (10441
ms)
[ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
Using 10000 agents and 1 frameworks with resource sharedness Shared
Added 1 frameworks in 243347ns
Added 10000 agents in 1.099817273secs
round 0 allocate() took 466.764704ms to make 10000 offers
round 1 allocate() took 466.218437ms to make 10000 offers
round 2 allocate() took 489.643342ms to make 10000 offers
round 3 allocate() took 470.636733ms to make 10000 offers
round 4 allocate() took 441.135047ms to make 10000 offers
round 5 allocate() took 439.438262ms to make 10000 offers
[       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49 (10693
ms)
...
```

1.10.x/master, before the preceding patch (r72508):
```
[ RUN      ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
Added 2000 agents in 26.693557ms
Added 2000 frameworks in 3.409660479secs
Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter 
Made 2000 allocations in 404.266006ms
Made 0 allocation in 102.640852ms
[       OK ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
(4373 ms)

[ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
Using 10000 agents and 1 frameworks with resource sharedness Regular
Added 1 frameworks in 13.767317ms
Added 10000 agents in 18.479510808secs
round 0 allocate() took 15.887923568secs to make 10000 offers
round 1 allocate() took 15.391255543secs to make 10000 offers
round 2 allocate() took 17.560065095secs to make 10000 offers
round 3 allocate() took 15.38378105secs to make 10000 offers
round 4 allocate() took 14.978897854secs to make 10000 offers
```

1.9.0:
```
[ RUN      ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
Added 2000 agents in 37.441968ms
Added 2000 frameworks in 4.032939233secs
Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter 
Made 2000 allocations in 330.437505ms
Made 0 allocation in 232.482817ms
[       OK ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
...
[ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
Using 10000 agents and 1 frameworks with resource sharedness Regular
Added 1 frameworks in 21.027254ms
Added 10000 agents in 806.232916ms
round 0 allocate() took 392.39282ms to make 10000 offers
round 1 allocate() took 366.960323ms to make 10000 offers
round 2 allocate() took 369.804604ms to make 10000 offers
round 3 allocate() took 361.787792ms to make 10000 offers
round 4 allocate() took 365.395693ms to make 10000 offers
round 5 allocate() took 394.293667ms to make 10000 offers
[       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48 (7939
ms)
[ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
Using 10000 agents and 1 frameworks with resource sharedness Shared
Added 1 frameworks in 232664ns
Added 10000 agents in 860.645269ms
round 0 allocate() took 394.275517ms to make 10000 offers
round 1 allocate() took 390.205739ms to make 10000 offers
round 2 allocate() took 391.286642ms to make 10000 offers
round 3 allocate() took 438.51943ms to make 10000 offers
round 4 allocate() took 399.460287ms to make 10000 offers
round 5 allocate() took 439.818163ms to make 10000 offers
[       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49 (8517
ms)
...
```


Thanks,

Andrei Sekretenko


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message