flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8458) Add the switch for keeping both the old mode and the new credit-based mode
Date Mon, 22 Jan 2018 11:11:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16334157#comment-16334157

ASF GitHub Bot commented on FLINK-8458:

Github user alpinegizmo commented on a diff in the pull request:

    --- Diff: docs/ops/config.md ---
    @@ -290,6 +290,12 @@ The following parameters configure Flink's JobManager and TaskManagers.
     - `taskmanager.network.numberOfBuffers` (deprecated, replaced by the three parameters
above): The number of buffers available to the network stack. This number determines how many
streaming data exchange channels a TaskManager can have at the same time and how well buffered
the channels are. If a job is rejected or you get a warning that the system has not enough
buffers available, increase this value (DEFAULT: **2048**). If set, it will be mapped to `taskmanager.network.memory.min`
and `taskmanager.network.memory.max` based on `taskmanager.memory.segment-size`.
    +- `taskmanager.network.memory.buffers-per-channel`: Number of network buffers to use
for each outgoing/incoming channel (subpartition/input channel). Especially in credit-based
flow control mode, it indicates how many credits are exclusive in each input channel. It should
be configured at least 2 for good performance. 1 buffer is for receving in-flight data in
the subpartition and 1 buffer is for parallel serialization. 
    --- End diff --
    This needs some clarification. Is taskmanager.network.memory.buffers-per-channel only
used in credit-based flow control mode? Does choosing a value greater than 2 provide any benefit?

> Add the switch for keeping both the old mode and the new credit-based mode
> --------------------------------------------------------------------------
>                 Key: FLINK-8458
>                 URL: https://issues.apache.org/jira/browse/FLINK-8458
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>            Reporter: zhijiang
>            Assignee: zhijiang
>            Priority: Major
>             Fix For: 1.5.0
> After the whole feature of credit-based flow control is done, we should add a config
parameter to switch on/off the new credit-based mode. To do so, we can roll back to the old
network mode for any expected risks.
> The parameter is defined as {{taskmanager.network.credit-based-flow-control.enabled}} and
the default value is true. This switch may be removed after next release.

This message was sent by Atlassian JIRA

View raw message