flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5791) Resource should be strictly matched when allocating for yarn
Date Fri, 03 Mar 2017 14:06:45 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15894414#comment-15894414
] 

ASF GitHub Bot commented on FLINK-5791:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3304#discussion_r104158286
  
    --- Diff: flink-yarn/src/main/java/org/apache/flink/yarn/YarnResourceManager.java ---
    @@ -376,4 +388,23 @@ private int generatePriority(ResourceProfile resourceProfile) {
     		}
     	}
     
    +	/**
    +	 * Get resource profile by given the container priority. It should always be got from
the mapping
    +	 * between resource profile and priority, otherwise it will throw the exception.
    +	 *
    +	 * @param priority The container priority used for distinguishing different resources.
    +	 * @return The resource profile corresponding with the priority.
    +	 * @throws Exception
    +	 */
    +	private ResourceProfile getResourceProfile(int priority) throws Exception{
    +		if (resourcePriorities.containsValue(priority)) {
    +			for (Map.Entry<ResourceProfile, Integer> entry : resourcePriorities.entrySet())
{
    +				if (entry.getValue() == priority) {
    +					return entry.getKey();
    +				}
    +			}
    +		}
    --- End diff --
    
    I think this should be changed. Why not allowing the `YarnResourceManger` to accept also
containers whose resource are larger than stated in the request? It should be ok for a slot
request to get a slot assigned whose resources exceed those of the request.


> Resource should be strictly matched when allocating for yarn
> ------------------------------------------------------------
>
>                 Key: FLINK-5791
>                 URL: https://issues.apache.org/jira/browse/FLINK-5791
>             Project: Flink
>          Issue Type: Improvement
>          Components: YARN
>            Reporter: shuai.xu
>            Assignee: shuai.xu
>              Labels: flip-6
>
> In flip6, for yarn mode, resource should be assigned as requested to avoid resource wasting
and OOM.
> 1. YarnResourceManager will request container according to ResourceProfile   in slot
request form JM.
> 2. RM will pass the ResourceProfile to TM for initializing its slots.
> 3. RM should match the slots offered by TM with SlotRequest from JM strictly.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message