hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
Date Fri, 08 Feb 2019 21:14:00 GMT
Eric Yang created YARN-9292:

             Summary: Implement logic to keep docker image consistent in application that
uses :latest tag
                 Key: YARN-9292
                 URL: https://issues.apache.org/jira/browse/YARN-9292
             Project: Hadoop YARN
          Issue Type: Sub-task
            Reporter: Eric Yang

Docker image with latest tag can run in YARN cluster without any validation in node managers.
If a image with latest tag is changed during containers launch. It might produce inconsistent
results between nodes. This is surfaced toward end of development for YARN-9184 to keep docker
image consistent within a job. One of the ideas to keep :latest tag consistent for a job,
is to use docker image command to figure out the image id and use image id to propagate to
rest of the container requests. There are some challenges to overcome:

 # The latest tag does not exist on the node where first container starts. The first container
will need to download the latest image, and find image ID. This can introduce lag time for
other containers to start.
 # If image id is used to start other container, container-executor may have problems to check
if the image is coming from a trusted source. Both image name and ID must be supply through
.cmd file to container-executor. However, hacker can supply incorrect image id and defeat
container-executor security checks.

If we can over come those challenges, it maybe possible to keep docker image consistent with
one application.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message