hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16312) Remove dumb-init from hadoop-runner image
Date Mon, 13 May 2019 16:25:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838676#comment-16838676
] 

Eric Yang commented on HADOOP-16312:
------------------------------------

{quote}Would you be so kind Eric Yang to explain it more details? Can you please give me an
example how scm process is respawned in the ozone compose clusters (where hadoop-runner is
used together with dumb-init).{quote}

That statement is flawed.  It does not respawn the program.  Instead, i meant reparenting
orphaned child processes and keep them alive or reap them.  It is possible to use:

{code}
CMD ["/bin/bash", "-c", "set -e && /opt/apache/ozone/bin/ozone"] 
{code}

to accomplish the same.  Kill the container will result in SIGKILL on orphaned child processes
to get a cleaner system.  We can talk about if it is ok to kill Ozone threads with SIGKILL
with some clean up strategy for empty or corrupted files or we want to code logic into signal
handling for Ozone threads.  I don't think we can do much to manage some JVM native threads,
and the current starter.sh will result in process threads that look like this:

{code}
/dumb-init
+--- /usr/bin/dumb-init
      +--- /bin/bash
            +--- java ...
{code}

Bash will not forward signal handling to Java.  Instead, the proper usage is:

{code}
exec java ...
{code}

The process tree will look like:

{code}
/dumb-init
+--- java ...
{code}

This will ensure the signal handling is correct.

> Remove dumb-init from hadoop-runner image
> -----------------------------------------
>
>                 Key: HADOOP-16312
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16312
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Elek, Marton
>            Priority: Major
>
> This issue is reported by [~eyang] in HDDS-1495.
> I think it's better to discuss under a separated issue as it's unrelated to HDDS-1495.
> The original problem description from [~eyang]
> {quote}Dumb-init  is one way to always run contaized program in the background and respawn
the program when program fails. This is poor man’s solution for keeping program alive.
> Cluster management software like Kubernetes or YARN have additional policy and logic
to start the same docker container on a different node. Therefore, Dumb-init is not recommended
for future Hadoop daemons instead allow cluster management software to make decision where
to start the container. Dumb-init for demonize docker container will be removed, and change
to use entrypoint.sh Docker provides -d flag to demonize foreground process. Most of the management
system built on top of Docker, (ie. Kitematic, Apache YARN, and Kubernetes) integrates with
Docker container at foreground to  aggregate stdout and stderr output of the containerized
program.
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message