mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Mahler <bmah...@apache.org>
Subject Re: Review Request 64428: Eliminated some copying of tasks / executors in agent re-registration.
Date Fri, 08 Dec 2017 00:03:02 GMT


> On Dec. 7, 2017, 10:15 p.m., Dmitry Zhuk wrote:
> > src/master/master.hpp
> > Line 713 (original), 713 (patched)
> > <https://reviews.apache.org/r/64428/diff/3/?file=1910802#file1910802line713>
> >
> >     shall we keep a default value?

I can update this to take an rvalue reference, at which point I can't have a default right?
That was originally why I removed it.


> On Dec. 7, 2017, 10:15 p.m., Dmitry Zhuk wrote:
> > src/master/master.cpp
> > Line 6700 (original), 6694 (patched)
> > <https://reviews.apache.org/r/64428/diff/3/?file=1910803#file1910803line6704>
> >
> >     btw, if large number of frameworks is a concern, we could also avoid double
lookup in `hashmap` with something like
> >     ```
> >     auto it = fameworks.find(task.framework_id());
> >     
> >     CHECK(it != frameworks.end());
> >     
> >     injectAllocationInfo(
> >         task.mutable_resources(),
> >         it->second);
> >     ```

Good point, we indeed have some excessive lookups, will look into the impact of this in a
separate patch.


> On Dec. 7, 2017, 10:15 p.m., Dmitry Zhuk wrote:
> > src/master/master.cpp
> > Lines 6770 (patched)
> > <https://reviews.apache.org/r/64428/diff/3/?file=1910803#file1910803line6780>
> >
> >     it seems that we can convert directly to `Resources` instead. `Slave`'s constructor
for some reason requires `vector<Resources>`, but then converts to `Resources` anyway

It was a little messy to do this since the registration path also uses the constructor, will
maybe look into it as a follow up.


> On Dec. 7, 2017, 10:15 p.m., Dmitry Zhuk wrote:
> > src/master/master.cpp
> > Lines 6837-6839 (patched)
> > <https://reviews.apache.org/r/64428/diff/3/?file=1910803#file1910803line6847>
> >
> >     we don't really need a conversion to `vector` for `updateSlaveFrameworks` if
performance here is a concern. it could take `RepeatedPtrField<FrameworkInfo>` as parameter
instead.
> >     this is also true for `addSlave`

Hm.. yeah, good point, I will explore that in a subsequent patch, see if it's helpful.


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64428/#review193167
-----------------------------------------------------------


On Dec. 7, 2017, 8:46 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64428/
> -----------------------------------------------------------
> 
> (Updated Dec. 7, 2017, 8:46 p.m.)
> 
> 
> Review request for mesos, Dmitry Zhuk and Michael Park.
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> See summary.
> 
> 
> Diffs
> -----
> 
>   src/master/master.hpp 2658312b0d10a72fefda68c7d3137b94afbd8249 
>   src/master/master.cpp 2fd66c072e9a194680d7653c664bd8a68ea1d2f0 
> 
> 
> Diff: https://reviews.apache.org/r/64428/diff/3/
> 
> 
> Testing
> -------
> 
> About a 10-20% improvement.
> 
> Before:
> Reregistered 2000 agents with a total of 100000 running tasks and 100000 completed tasks
in 2.977767303secs
> Reregistered 2000 agents with a total of 200000 running tasks and 0 completed tasks in
5.642229947secs
> Reregistered 20000 agents with a total of 100000 running tasks and 0 completed tasks
in 6.966318315secs
> 
> After:
> Reregistered 2000 agents with a total of 100000 running tasks and 100000 completed tasks
in 2.591612374secs
> Reregistered 2000 agents with a total of 200000 running tasks and 0 completed tasks in
4.52633125secs
> Reregistered 20000 agents with a total of 100000 running tasks and 0 completed tasks
in 6.449984263secs
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message