flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Yao (Jira)" <j...@apache.org>
Subject [jira] [Assigned] (FLINK-14362) Change DefaultSchedulingResultPartition to return correct partition state
Date Fri, 15 Nov 2019 14:36:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-14362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gary Yao reassigned FLINK-14362:
--------------------------------

    Assignee: Zhu Zhu

> Change DefaultSchedulingResultPartition to return correct partition state
> -------------------------------------------------------------------------
>
>                 Key: FLINK-14362
>                 URL: https://issues.apache.org/jira/browse/FLINK-14362
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Coordination
>    Affects Versions: 1.10.0
>            Reporter: Zhu Zhu
>            Assignee: Zhu Zhu
>            Priority: Major
>             Fix For: 1.10.0
>
>
> Currently {{DefaultSchedulingResultPartition#getState()}} returns the state of partitions
based on the partition producer's state. The state is used to make scheduling decision.
> However, it does not correctly reflect the true state of a partition.
> For example, when producer task turns to RUNNING but has not produced any data yet, it's
consumers should not be scheduled to reduce unnecessary resource cost in lazy scheduling mode.
However, the partition state will be RUNNING in {{DefaultSchedulingResultPartition}} and will
trigger the scheduling of its consumers.
> I'd propose to change the enums in {{ResultPartitionState}} to be:
> * CREATED // partition is just created or is just reset
> * CONSUMABLE // pipelined partition has data produced or blocking partition's parent
result finishes. Corresponds to IntermediateResultPartition#isConsumable.
> The CONSUMABLE state is what the scheduler cares to make scheduling decisions.
> This change also relieves LazyFromSourcesSchedulingStrategy/InputDependencyConstraintChecker
from partition states management.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message