flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-11082) Increase backlog only if it is available for consumption
Date Wed, 06 Mar 2019 09:20:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-11082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

ASF GitHub Bot updated FLINK-11082:
    Labels: pull-request-available  (was: )

> Increase backlog only if it is available for consumption
> --------------------------------------------------------
>                 Key: FLINK-11082
>                 URL: https://issues.apache.org/jira/browse/FLINK-11082
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Runtime / Network
>    Affects Versions: 1.5.6, 1.6.3, 1.7.1, 1.8.0
>            Reporter: zhijiang
>            Assignee: zhijiang
>            Priority: Major
>              Labels: pull-request-available
> The backlog should indicate how many buffers are available in subpartition for downstream's 
consumption. The availability is considered from two factors. One is {{BufferConsumer}}
finished, and the other is flush triggered.
> In current implementation, when the {{BufferConsumer}} is added into the subpartition,
then the backlog is increased as a result, but this {{BufferConsumer}} is not yet available
for network transport.
> Furthermore, the backlog would affect requesting floating buffers on downstream side.
That means some floating buffers are fetched in advance but not be used for long time, so
the floating buffers are not made use of efficiently.
> We found this scenario extremely for rebalance selector on upstream side, so we want
to change when to increase backlog by finishing {{BufferConsumer}} or flush triggered.

This message was sent by Atlassian JIRA

View raw message