mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mesos Reviewbot Windows <revi...@mesos.apache.org>
Subject Re: Review Request 67965: Optimized ranges subtraction operation.
Date Wed, 25 Jul 2018 01:23:01 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67965/#review206435
-----------------------------------------------------------



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67965']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1978/mesos-review-67965

Relevant logs:

- [libprocess-tests-cmake-stdout.log](http://dcos-win.westus.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/1978/mesos-review-67965/logs/libprocess-tests-cmake-stdout.log):

```
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(1520):
warning C4244: '+=': conversion from '__int64' to 'uint32_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(1689):
warning C4244: '+=': conversion from '__int64' to 'uint32_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2273):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2280):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2292):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2301):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2401):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         cl : Command line warning D9002: ignoring unknown option '-fPIC' [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(1415):
warning C4244: '+=': conversion from '__int64' to 'uint32_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(1520):
warning C4244: '+=': conversion from '__int64' to 'uint32_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(1689):
warning C4244: '+=': conversion from '__int64' to 'uint32_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2273):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2280):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2292):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2301):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]
         d:\dcos\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2\http_parser.c(2401):
warning C4244: '=': conversion from '__int64' to 'uint16_t', possible loss of data [D:\DCOS\mesos\3rdparty\http_parser-2.6.2\src\http_parser-2.6.2-build\http_parser.vcxproj]
[D:\DCOS\mesos\3rdparty\http_parser-2.6.2.vcxproj]


       "D:\DCOS\mesos\3rdparty\libprocess\src\tests\libprocess-tests.vcxproj" (default target)
(1) ->
       "D:\DCOS\mesos\3rdparty\libprocess\src\tests\benchmarks.vcxproj" (default target) (6)
->
       (ClCompile target) -> 
         d:\dcos\mesos\mesos\3rdparty\libprocess\src\mpsc_linked_queue.hpp(181): error C2327:
'process::MpscLinkedQueue<T>::tail': is not a type name, static, or enumerator (compiling
source file D:\DCOS\mesos\mesos\3rdparty\libprocess\src\tests\benchmarks.cpp) [D:\DCOS\mesos\3rdparty\libprocess\src\tests\benchmarks.vcxproj]
         d:\dcos\mesos\mesos\3rdparty\libprocess\src\mpsc_linked_queue.hpp(181): error C2065:
'tail': undeclared identifier (compiling source file D:\DCOS\mesos\mesos\3rdparty\libprocess\src\tests\benchmarks.cpp)
[D:\DCOS\mesos\3rdparty\libprocess\src\tests\benchmarks.vcxproj]
         d:\dcos\mesos\mesos\3rdparty\libprocess\src\mpsc_linked_queue.hpp(181): error C2327:
'process::MpscLinkedQueue<std::string>::tail': is not a type name, static, or enumerator
(compiling source file D:\DCOS\mesos\mesos\3rdparty\libprocess\src\tests\benchmarks.cpp) [D:\DCOS\mesos\3rdparty\libprocess\src\tests\benchmarks.vcxproj]

    18 Warning(s)
    3 Error(s)

Time Elapsed 00:02:13.47
```

- Mesos Reviewbot Windows


On July 25, 2018, 12:23 a.m., Meng Zhu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67965/
> -----------------------------------------------------------
> 
> (Updated July 25, 2018, 12:23 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Gastón Kleiman, and Vinod Kone.
> 
> 
> Bugs: MESOS-9086
>     https://issues.apache.org/jira/browse/MESOS-9086
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> The current ranges subtraction uses boost IntervalSet.
> Based on the profiling result of MESOS-8989, the ranges
> subtraction operation is about 2~3 times more expensive
> than that of addition. The conversion cost from Ranges
> to IntervalSet may be the culprit.
> 
> This patch optimizes the ranges subtraction operation by
> converting Ranges to a vector of internal sub-ranges, sorting
> the vector based on sub-range start and then applying a
> one-pass algorithm similar to that of addition.
> 
> 
> Diffs
> -----
> 
>   src/common/values.cpp afe4137f82115dd4ec9967b5eba16a1dd15401c8 
>   src/tests/values_tests.cpp 2f5abedfd461c114d03f5e941d81ebe414188033 
>   src/v1/values.cpp d2c31f6c91998382dec1d8834b40613013716cdd 
> 
> 
> Diff: https://reviews.apache.org/r/67965/diff/3/
> 
> 
> Testing
> -------
> 
> make check
> 
> Benchmarking:
> 
> tl:dr: more than 80% performance improvment across board. Performance of subtraction
is now on par or better than the addition, especially when there are a large number of sub-ranges.
> 
> Build with -O2 optimization, ran on a multicore machine with peak frequency at 2.2GHz:
> 
> Took 1.617706ms to perform 1000 'a += b' operations on ports:[1-6, 11-16, 21-26...91-96]
and ports:[3-8, 13-18, 23-28..., 93-98] with 10 sub-ranges
> Took 1.607999ms to perform 1000 'a -= b' operations on ports:[1-6, 11-16, 21-26...91-96]
and ports:[3-8, 13-18, 23-28..., 93-98] with 10 sub-ranges
> Took 3.094113ms to perform 1000 'a + b' operations on ports:[1-6, 11-16, 21-26...91-96]
and ports:[3-8, 13-18, 23-28..., 93-98] with 10 sub-ranges
> Took 3.152291ms to perform 1000 'a - b' operations on ports:[1-6, 11-16, 21-26...91-96]
and ports:[3-8, 13-18, 23-28..., 93-98] with 10 sub-ranges
> 
> Took 14.908924ms to perform 1000 'a += b' operations on ports:[1-6, 11-16, 21-26...991-996]
and ports:[3-8, 13-18, 23-28..., 993-998] with 100 sub-ranges
> Took 13.564767ms to perform 1000 'a -= b' operations on ports:[1-6, 11-16, 21-26...991-996]
and ports:[3-8, 13-18, 23-28..., 993-998] with 100 sub-ranges
> Took 25.523443ms to perform 1000 'a + b' operations on ports:[1-6, 11-16, 21-26...991-996]
and ports:[3-8, 13-18, 23-28..., 993-998] with 100 sub-ranges
> Took 24.871216ms to perform 1000 'a - b' operations on ports:[1-6, 11-16, 21-26...991-996]
and ports:[3-8, 13-18, 23-28..., 993-998] with 100 sub-ranges
> 
> Took 234.22483ms to perform 1000 'a += b' operations on ports:[1-6, 11-16, 21-26...9991-9996]
and ports:[3-8, 13-18, 23-28..., 9993-9998] with 1000 sub-ranges
> Took 144.417381ms to perform 1000 'a -= b' operations on ports:[1-6, 11-16, 21-26...9991-9996]
and ports:[3-8, 13-18, 23-28..., 9993-9998] with 1000 sub-ranges
> Took 322.548021ms to perform 1000 'a + b' operations on ports:[1-6, 11-16, 21-26...9991-9996]
and ports:[3-8, 13-18, 23-28..., 9993-9998] with 1000 sub-ranges
> Took 227.835441ms to perform 1000 'a - b' operations on ports:[1-6, 11-16, 21-26...9991-9996]
and ports:[3-8, 13-18, 23-28..., 9993-9998] with 1000 sub-ranges
> 
> 
> Thanks,
> 
> Meng Zhu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message