mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <bmah...@apache.org>
Subject Re: Review Request 72509: Got rid of tracking reserved offered/allocated in roles tree.
Date Thu, 14 May 2020 18:05:51 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72509/#review220767
-----------------------------------------------------------




src/master/allocator/mesos/hierarchical.hpp
Lines 787 (patched)
<https://reviews.apache.org/r/72509/#comment309459>

    Seems cleaner to have everything tracked in the role tree rather than have special case
root items tracked up in the allocator here.
    
    Are we gaining a lot with this?


- Benjamin Mahler


On May 14, 2020, 2:42 p.m., Andrei Sekretenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72509/
> -----------------------------------------------------------
> 
> (Updated May 14, 2020, 2:42 p.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Bugs: MESOS-10128
>     https://issues.apache.org/jira/browse/MESOS-10128
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Got rid of tracking reserved offered/allocated in roles tree.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp 6454cdaa19f776365df34ecf83114f0d6fa20f27

>   src/master/allocator/mesos/hierarchical.cpp 5fe9ffcb518b8427d663ddae43e550795d290e3c

> 
> 
> Diff: https://reviews.apache.org/r/72509/diff/1/
> 
> 
> Testing
> -------
> 
> `make check`
> 
> Ran hierarchical allocator benchmarks, compared with current results on master and 1.9.0
release.
> Below are results that improved significantly (compared to 1.10.x branch root) after
this fix.
> 
> 1.10.x/master, before the preceding patch (r72508):
> ```
> [ RUN      ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
> Added 2000 agents in 26.693557ms
> Added 2000 frameworks in 3.409660479secs
> Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter 
> Made 2000 allocations in 404.266006ms
> Made 0 allocation in 102.640852ms
> [       OK ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
(4373 ms)
> 
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
> Using 10000 agents and 1 frameworks with resource sharedness Regular
> Added 1 frameworks in 13.767317ms
> Added 10000 agents in 18.479510808secs
> round 0 allocate() took 15.887923568secs to make 10000 offers
> round 1 allocate() took 15.391255543secs to make 10000 offers
> round 2 allocate() took 17.560065095secs to make 10000 offers
> round 3 allocate() took 15.38378105secs to make 10000 offers
> round 4 allocate() took 14.978897854secs to make 10000 offers
> (killed manually)
> ```
> 
> After fix:
> ```
> [ RUN      ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
> Added 2000 agents in 28.968777ms
> Added 2000 frameworks in 3.254205437secs
> Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter 
> Made 2000 allocations in 190.023349ms
> Made 0 allocation in 81.404659ms
> [       OK ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
(3972 ms)
> 
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
> Using 10000 agents and 1 frameworks with resource sharedness Regular
> Added 1 frameworks in 265991ns
> Added 10000 agents in 1.088327374secs
> round 0 allocate() took 398.34303ms to make 10000 offers
> round 1 allocate() took 436.728365ms to make 10000 offers
> round 2 allocate() took 431.187768ms to make 10000 offers
> round 3 allocate() took 461.27924ms to make 10000 offers
> round 4 allocate() took 426.078997ms to make 10000 offers
> round 5 allocate() took 478.756617ms to make 10000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
(10441 ms)
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
> Using 10000 agents and 1 frameworks with resource sharedness Shared
> Added 1 frameworks in 243347ns
> Added 10000 agents in 1.099817273secs
> round 0 allocate() took 466.764704ms to make 10000 offers
> round 1 allocate() took 466.218437ms to make 10000 offers
> round 2 allocate() took 489.643342ms to make 10000 offers
> round 3 allocate() took 470.636733ms to make 10000 offers
> round 4 allocate() took 441.135047ms to make 10000 offers
> round 5 allocate() took 439.438262ms to make 10000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
(10693 ms)
> ...
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/132
> Using 50000 agents and 500 frameworks with resource sharedness Regular
> Added 500 frameworks in 6.455113ms
> Added 50000 agents in 5.48064759secs
> round 0 allocate() took 3.086837277secs to make 50000 offers
> round 1 allocate() took 3.172595586secs to make 50000 offers
> round 2 allocate() took 3.378091827secs to make 50000 offers
> round 3 allocate() took 3.394915982secs to make 50000 offers
> round 4 allocate() took 3.724787378secs to make 50000 offers
> round 5 allocate() took 3.315077294secs to make 50000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/132
(57814 ms)
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/133
> Using 50000 agents and 500 frameworks with resource sharedness Shared
> Added 500 frameworks in 171.741816ms
> Added 50000 agents in 22.066628935secs
> round 0 allocate() took 18.788494416secs to make 50000 offers
> round 1 allocate() took 17.854696568secs to make 50000 offers
> round 2 allocate() took 16.215714459secs to make 50000 offers
> round 3 allocate() took 16.195078082secs to make 50000 offers
> round 4 allocate() took 15.427792123secs to make 50000 offers
> round 5 allocate() took 15.297172106secs to make 50000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/133
(165269 ms)
> ```
> 
> 
> 1.9.0:
> ```
> [ RUN      ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
> Added 2000 agents in 37.441968ms
> Added 2000 frameworks in 4.032939233secs
> Quota run setup: 2000 agents, 1000 roles, 2000 frameworks, with drf sorter 
> Made 2000 allocations in 330.437505ms
> Made 0 allocation in 232.482817ms
> [       OK ] NonQuotaVsQuotaParam/BENCHMARK_HierarchicalAllocator_WithNonQuotaVsQuotaParam.NonQuotaVsQuota/5
> ...
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
> Using 10000 agents and 1 frameworks with resource sharedness Regular
> Added 1 frameworks in 21.027254ms
> Added 10000 agents in 806.232916ms
> round 0 allocate() took 392.39282ms to make 10000 offers
> round 1 allocate() took 366.960323ms to make 10000 offers
> round 2 allocate() took 369.804604ms to make 10000 offers
> round 3 allocate() took 361.787792ms to make 10000 offers
> round 4 allocate() took 365.395693ms to make 10000 offers
> round 5 allocate() took 394.293667ms to make 10000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/48
(7939 ms)
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
> Using 10000 agents and 1 frameworks with resource sharedness Shared
> Added 1 frameworks in 232664ns
> Added 10000 agents in 860.645269ms
> round 0 allocate() took 394.275517ms to make 10000 offers
> round 1 allocate() took 390.205739ms to make 10000 offers
> round 2 allocate() took 391.286642ms to make 10000 offers
> round 3 allocate() took 438.51943ms to make 10000 offers
> round 4 allocate() took 399.460287ms to make 10000 offers
> round 5 allocate() took 439.818163ms to make 10000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/49
(8517 ms)
> ...
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/132
> Using 50000 agents and 500 frameworks with resource sharedness Regular
> Added 500 frameworks in 6.473628ms
> Added 50000 agents in 5.121473939secs
> round 0 allocate() took 2.830646066secs to make 50000 offers
> round 1 allocate() took 3.162636227secs to make 50000 offers
> round 2 allocate() took 2.916963087secs to make 50000 offers
> round 3 allocate() took 3.308795663secs to make 50000 offers
> round 4 allocate() took 2.923405388secs to make 50000 offers
> round 5 allocate() took 2.830233693secs to make 50000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/132
(49633 ms)
> [ RUN      ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/133
> Using 50000 agents and 500 frameworks with resource sharedness Shared
> Added 500 frameworks in 222.196039ms
> Added 50000 agents in 21.331638546secs
> round 0 allocate() took 18.571915447secs to make 50000 offers
> round 1 allocate() took 18.686376839secs to make 50000 offers
> round 2 allocate() took 18.339303874secs to make 50000 offers
> round 3 allocate() took 18.629399242secs to make 50000 offers
> round 4 allocate() took 18.664419996secs to make 50000 offers
> round 5 allocate() took 18.970253393secs to make 50000 offers
> [       OK ] AllResources/HierarchicalAllocations_BENCHMARK_Test.PersistentVolumes/133
(173206 ms)
> ```
> 
> 
> Thanks,
> 
> Andrei Sekretenko
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message