mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Mann <g...@mesosphere.io>
Subject Re: Review Request 65518: Reaped the container process directly in Docker executor.
Date Thu, 15 Feb 2018 00:03:46 GMT


> On Feb. 14, 2018, 5:49 p.m., Andrei Budnik wrote:
> > src/docker/executor.cpp
> > Lines 304 (patched)
> > <https://reviews.apache.org/r/65518/diff/4/?file=1958430#file1958430line304>
> >
> >     This case means `docker stop` is in progress (`killed == true`): https://github.com/apache/mesos/blob/e36e3b9989b1097dca9597097dd51d24aaba48fa/src/docker/executor.cpp#L473-L525.
The container might still be running. If `docker stop` fails or hangs for more than 3 seconds,
the executor will terminate, but the docker container might be running.

Could you elaborate here, I don't quite understand? In this case, we know that `container.pid.isNone()
== true`, which means that the `Docker::inspect()` call returned successfully, but no running
process was found for that container. The conditional above tests for `if (!killed)`, but
we don't know anything about the value of `killed` here.


- Greg


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65518/#review197567
-----------------------------------------------------------


On Feb. 13, 2018, 3:20 a.m., Qian Zhang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65518/
> -----------------------------------------------------------
> 
> (Updated Feb. 13, 2018, 3:20 a.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Gilbert Song, Greg Mann, and Vinod Kone.
> 
> 
> Bugs: MESOS-8488
>     https://issues.apache.org/jira/browse/MESOS-8488
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> Due to a Docker issue (https://github.com/moby/moby/issues/33820),
> Docker daemon can fail to catch a container exit, i.e., the container
> process has already exited but the command `docker ps` shows the
> container still running, this will lead to the "docker run" command
> that we execute in Docker executor never returning, and it will also
> cause the `docker stop` command takes no effect, i.e., it will return
> without error but `docker ps` shows the container still running, so
> the task will stuck in `TASK_KILLING` state.
> 
> To workaround this Docker issue, in this patch we made Docker executor
> reaps the container process directly so Docker executor will be notified
> once the container process exits.
> 
> 
> Diffs
> -----
> 
>   src/docker/executor.cpp e4c53d558e414e50b1c429fba8e31e504c63744a 
> 
> 
> Diff: https://reviews.apache.org/r/65518/diff/4/
> 
> 
> Testing
> -------
> 
> sudo make check
> 
> 
> Thanks,
> 
> Qian Zhang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message