flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Yao (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-14400) Shrink the scope of MemoryManager from TaskExecutor to slot
Date Thu, 16 Jan 2020 16:18:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-14400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017076#comment-17017076

Gary Yao commented on FLINK-14400:

master: 9fed0ddc5bc015f98246a2d8d9adbe5fb2b91ba4

> Shrink the scope of MemoryManager from TaskExecutor to slot 
> ------------------------------------------------------------
>                 Key: FLINK-14400
>                 URL: https://issues.apache.org/jira/browse/FLINK-14400
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Coordination, Runtime / Task
>            Reporter: Andrey Zagrebin
>            Assignee: Andrey Zagrebin
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.10.0
>          Time Spent: 20m
>  Remaining Estimate: 0h
> MemoryManager currently manages the memory bookkeeping for all slots/tasks inside one TaskExecutor.
For better abstraction and isolation of slots, we can shrink its scope and make it per slot.
The memory limits are fixed now per slot at the moment of slot creation. All operators, sharing
the slot, will get their fixed fractional limits.
> In future, we might make it possible for operators to over-allocate beyond their fraction
limit if there is some available free memory in the slot but it should be possible to reclaim
the over-allocated memory at any time if other operator decides to claim its fair share within
its limit.

This message was sent by Atlassian Jira

View raw message