flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Yao (Jira)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-14058) FLIP-53 Fine Grained Operator Resource Management
Date Tue, 28 Jan 2020 10:20:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-14058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Gary Yao updated FLINK-14058:
    Release Note: 
Config options `table.exec.resource.external-buffer-memory`,
`table.exec.resource.hash-agg.memory`, `table.exec.resource.hash-join.memory`,
and `table.exec.resource.sort.memory` have been deprecated. Beginning from Flink
1.10, these config options are interpreted as weight hints instead of absolute
memory requirements. Flink choses sensible default weight hints, which should
not need adjustment by users.

The memory configs for table operators are no longer absolute memory requirements but weight
hints now. This means that the actual managed memory allocated can be smaller or larger than
the config values, which depends on the actual slot managed memory capacity.
The affected configs are table.exec.resource.external-buffer-memory, table.exec.resource.hash-agg.memory,
table.exec.resource.hash-join.memory, and table.exec.resource.sort.memory.
This change ensures that operators do not over allocate memory or leave available memory unallocated.
It helps the task to be more stable if the slot managed memory is limited, or more efficient
if the slot has adequate managed memory.

> FLIP-53 Fine Grained Operator Resource Management
> -------------------------------------------------
>                 Key: FLINK-14058
>                 URL: https://issues.apache.org/jira/browse/FLINK-14058
>             Project: Flink
>          Issue Type: New Feature
>          Components: Runtime / Coordination
>    Affects Versions: 1.9.0
>            Reporter: Xintong Song
>            Assignee: Zhu Zhu
>            Priority: Major
>              Labels: Umbrella
>             Fix For: 1.10.0
> This is the umbrella issue of 'FLIP-53: Fine Grained Operator Resource Management'.

This message was sent by Atlassian Jira

View raw message