mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <bmah...@apache.org>
Subject Re: Review Request 48906: Added helper function to get non-scarce resources.
Date Wed, 22 Jun 2016 21:08:51 GMT


> On June 21, 2016, 10:36 p.m., Benjamin Mahler wrote:
> > include/mesos/resources.hpp, lines 252-256
> > <https://reviews.apache.org/r/48906/diff/1/?file=1423388#file1423388line252>
> >
> >     The Resources abstraction shouldn't need to know anything about scarce resources,
you can just use the 'filter' function above if you want to perform some custom filtering
in the caller.
> 
> Guangya Liu wrote:
>     Hi Ben, so you mean that we still need to keep the function of `isNonScarce` here
but only remove `nonScarce`? 
>     
>     Can you please explain more for why do you think the resourcs abstraction cannot
expose anything of `scarce` resources? In my understadnig, I think that we should put `scarce`
resources at same level as `revocable`, `reserved` resources etc, all of them can be treated
as `different kind of resources types`, so seems putting this function here is reasonable?
> 
> Benjamin Mahler wrote:
>     Revocable and reservation information are properties of the resource. Scarcity is
depdendent on the environment rather than being a property of the resource itself. If you
have a lot of GPUs in a pool, they are not scarce. If you have few GPUs in a pool, then they
are scarce.
> 
> Guangya Liu wrote:
>     I think that for revocable and reservations, both of them can be configured by cluster
admininstrator or framework developer.
>     
>     The cluster administrator can define the revocable resources via the resource estimator
for now.
>     
>     The reservation can be configured by admin or framework developer when starting the
agent or using http endpoint or framework. Here we can configure some resources from non reserved
to reserved resources. If we did some reserve operations, then the resources will becomre
reserved resources, otherwise, it will not be reserved resource.
>     
>     Similar with above, the cluster admin can also configure the scarce resources when
master start via a flag, it is also configured by cluster admin.
>     
>     So I think that revocable, reserved and scarce can all be treated as resource property?

In the discussions I've started around "scarcity", I have used the word "scarcity" to describe
when a resource is only present on a small number of machines. This means that the "scarcity"
of the resource is not determined by the operator or by the framework, but is rather a property
of the pool of resources in which the resource is present. If all 4 machines in a cluster
happen to have 1 GPU, then GPUs are not "scarce". If 1000 machines join the cluster without
GPUs, now the (4) GPUs fit the definition of a "scarce" resource.

So I'm confused as to why you would want to have the operator set the "scarcity" property
on a resource. What advantage would there be of doing this?

Also note that the Resources abstraction should know nothing about fairness or fairness exclusion.


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48906/#review138958
-----------------------------------------------------------


On June 18, 2016, 1:04 p.m., Guangya Liu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48906/
> -----------------------------------------------------------
> 
> (Updated June 18, 2016, 1:04 p.m.)
> 
> 
> Review request for mesos and Benjamin Mahler.
> 
> 
> Bugs: MESOS-5621
>     https://issues.apache.org/jira/browse/MESOS-5621
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Added helper function to get non-scarce resources.
> 
> 
> Diffs
> -----
> 
>   include/mesos/resources.hpp a557e97c65194d4aad879fb88d8edefd1c95b8d8 
>   include/mesos/v1/resources.hpp a5ba8fec4c9c3643646308f75a4b28cefe0b3df3 
>   src/common/resources.cpp f6ff92b591c15bc8e93fd85e1896349c3a7bb968 
>   src/tests/resources_tests.cpp dc12bd8f1e2da6972bc8aed598811c55d664036e 
>   src/v1/resources.cpp 8c3f2d1c1529915a59d47fe37bb3fc7a3267079a 
> 
> Diff: https://reviews.apache.org/r/48906/diff/
> 
> 
> Testing
> -------
> 
> make
> make check
> 
> [==========] Running 1 test from 1 test case.
> [----------] Global test environment set-up.
> [----------] 1 test from ResourcesTest
> [ RUN      ] ResourcesTest.NonScarce
> [       OK ] ResourcesTest.NonScarce (1 ms)
> [----------] 1 test from ResourcesTest (1 ms total)
> 
> [----------] Global test environment tear-down
> [==========] 1 test from 1 test case ran. (14 ms total)
> [  PASSED  ] 1 test.
> 
> 
> Thanks,
> 
> Guangya Liu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message