flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2954) Not able to pass custom environment variables in cluster to processes that spawning TaskManager
Date Thu, 26 Nov 2015 12:26:11 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028668#comment-15028668
] 

ASF GitHub Bot commented on FLINK-2954:
---------------------------------------

Github user rmetzger commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1409#discussion_r45973217
  
    --- Diff: flink-yarn-tests/src/main/java/org/apache/flink/yarn/UtilsTest.java ---
    @@ -97,6 +98,18 @@ public void tooMuchCutoff() {
     		Assert.assertEquals(0, Utils.calculateHeapSize(4000, conf));
     	}
     
    +	@Test
    +	public void testGetEnvironmentVariables() {
    +		Configuration testConf = new Configuration();
    +		testConf.setString("yarn.application-master.env.LD_LIBRARY_PATH", "/usr/lib/native");
    +
    +		Map<String, String> res = Utils.getEnvironmentVariables("yarn.application-master.env.",
testConf);
    +
    +		Assert.assertEquals(1, res.size());
    +		Map.Entry<String, String> entry = res.entrySet().iterator().next();
    +		Assert.assertEquals("LD_LIBRARY_PATH", entry.getKey());
    +		Assert.assertEquals("/usr/lib/native", entry.getValue());
    +	}
    --- End diff --
    
    I think its okay to assume that the method will work for all prefix strings.
    This test is only about the method for extracting the variables.


> Not able to pass custom environment variables in cluster to processes that spawning TaskManager
> -----------------------------------------------------------------------------------------------
>
>                 Key: FLINK-2954
>                 URL: https://issues.apache.org/jira/browse/FLINK-2954
>             Project: Flink
>          Issue Type: Bug
>          Components: Command-line client, Distributed Runtime
>    Affects Versions: 0.10.0
>            Reporter: Jian Jiang
>            Assignee: Robert Metzger
>            Priority: Critical
>             Fix For: 1.0.0
>
>
> There are programs that rely on custom environment variables. In hadoop mapreduce job
we can use -Dmapreduce.map.env and - Dmapreduce.reduce.env to do pass them. Similarly in Spark
> we can use --conf 'spark.executor.XXX=value for XXX'. There is no such feature yet in
Flink.
> This has given Flink a serious disadvantage when customers need such feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message