hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Remi Catherinot (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-5714) ContainerExecutor does not order environment map
Date Thu, 06 Oct 2016 21:07:20 GMT
Remi Catherinot created YARN-5714:

             Summary: ContainerExecutor does not order environment map
                 Key: YARN-5714
                 URL: https://issues.apache.org/jira/browse/YARN-5714
             Project: Hadoop YARN
          Issue Type: Bug
          Components: nodemanager
    Affects Versions: 3.0.0-alpha1, 2.6.4, 2.7.3, 2.5.2, 2.4.1
         Environment: all (linux and windows alike)
            Reporter: Remi Catherinot
            Priority: Trivial

when dumping the launch container script, environment variables are dumped based on the order
internally used by the map implementation (hash based). It does not take into consideration
that some env varibales may refer each other, and so that some env variables must be declared
before those referencing them.

In my case, i ended having LD_LIBRARY_PATH which was depending on HADOOP_COMMON_HOME being
dumped before HADOOP_COMMON_HOME. Thus it had a wrong value and so native libraries weren't
loaded. jobs were running but not at their best. This is just a use case falling into that
bug, but i'm sure others may happen as well.

I already have a patch running in my production environment, i just estimate to 5 days for
packing the path in the right fashion for JIRA + try my best to add tests.

Note : the patch is not OS aware with a default empty implementation. I will only implement
the unix version on a 1st release. I'm not used to windows env variables syntax so it will
take me more time/research for it.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message