mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Peach <jpe...@apache.org>
Subject Re: Review Request 67914: Updated XFS disk isolator to reclaim project IDs after disk GC.
Date Wed, 18 Jul 2018 20:51:57 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67914/#review206203
-----------------------------------------------------------




src/slave/containerizer/mesos/isolators/xfs/disk.cpp
Line 495 (original), 504 (patched)
<https://reviews.apache.org/r/67914/#comment289095>

    I'm a little concerned that we could exhaust the project IDs (we default to 5000 IDs).
What do you think about adding a metric for the number of available project IDs?



src/slave/containerizer/mesos/isolators/xfs/disk.cpp
Lines 547 (patched)
<https://reviews.apache.org/r/67914/#comment289094>

    Let's make this message symmetric with the one we emit when we assign:
    
    ```
    LOG(INFO) << "Reclaimed project " << stringify(projectId.get()) << "
from '"
              << containerConfig.directory() << "'";
    ```



src/slave/containerizer/mesos/isolators/xfs/disk.cpp
Lines 549 (patched)
<https://reviews.apache.org/r/67914/#comment289093>

    If possible, I think this is simple enough to inline into the check loop



src/slave/flags.cpp
Lines 1338 (patched)
<https://reviews.apache.org/r/67914/#comment289092>

    Rather than introducing a new flag, lets use `container_disk_watch_interval` or `disk_watch_interval`
... probably the former?



src/tests/containerizer/xfs_quota_tests.cpp
Lines 1197 (patched)
<https://reviews.apache.org/r/67914/#comment289099>

    For consistency with other tests, could we call this `ProjectIdReclaiming`?
    
    Can you please add a short comment explaining the purpose of the test?



src/tests/containerizer/xfs_quota_tests.cpp
Lines 1319 (patched)
<https://reviews.apache.org/r/67914/#comment289100>

    Is this re-using the futures an OK thing to do? If they are already ready from the previous
task launch, won't they still be ready when we await them here?



src/tests/containerizer/xfs_quota_tests.cpp
Lines 1330 (patched)
<https://reviews.apache.org/r/67914/#comment289096>

    This can be `EXPECT_EQ`.



src/tests/containerizer/xfs_quota_tests.cpp
Lines 1332 (patched)
<https://reviews.apache.org/r/67914/#comment289097>

    Is the link the `latest` link? Can you add an explanatory comment?



src/tests/containerizer/xfs_quota_tests.cpp
Lines 1333 (patched)
<https://reviews.apache.org/r/67914/#comment289098>

    This can me `EXPECT_SOME_EQ`.


- James Peach


On July 13, 2018, 9:59 p.m., Ilya Pronin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67914/
> -----------------------------------------------------------
> 
> (Updated July 13, 2018, 9:59 p.m.)
> 
> 
> Review request for mesos and James Peach.
> 
> 
> Bugs: MESOS-9007
>     https://issues.apache.org/jira/browse/MESOS-9007
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Currently upon container destruction its project ID is unallocated by
> the isolator and removed from the container work directory. However due
> to API limitations we can't unset project IDs on symlinks that may exist
> inside the directory. Because of that the project may still exist until
> the container directory is garbage collected. If the project ID is
> reused for a new container, any lingering symlinks that still have that
> project ID will contribute to disk usage of the new container. Typically
> symlinks don't take much space, but still this leads to inaccuracy in
> disk space usage accounting.
> 
> This patch postpones project ID reclaiming until sandbox GC time. The
> isolator periodically checks if sandboxes of terminated containers still
> exist and deallocates project IDs of the ones that were removed. This
> mechanism can be improved in the future if we introduce a way for the
> isolators to learn about disk GCs.
> 
> 
> Diffs
> -----
> 
>   src/slave/containerizer/mesos/isolators/xfs/disk.hpp 0891f7709aa4f98758a727856d58e6177d46adca

>   src/slave/containerizer/mesos/isolators/xfs/disk.cpp 25f52a43b34b141bdaf7c448817423cf4264e22a

>   src/slave/flags.hpp eeb9708f9ec76d83b6719541f4a012544c7c0cbe 
>   src/slave/flags.cpp 58cdc0f1100fe244e5bf1036e1ccf39478d5d478 
>   src/tests/containerizer/xfs_quota_tests.cpp bee0c2db7adc1dc7d774f9152931130093fe5b50

> 
> 
> Diff: https://reviews.apache.org/r/67914/diff/2/
> 
> 
> Testing
> -------
> 
> Added `ROOT_XFS_QuotaTest.ProjectIDReclaiming` test that verifies that project ID is
reclaimed and reused after sandbox GC. Ran `sudo make check`.
> 
> 
> Thanks,
> 
> Ilya Pronin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message