flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alpinegizmo <...@git.apache.org>
Subject [GitHub] flink pull request #5317: [FLINK-8458] Add the switch for keeping both the o...
Date Mon, 22 Jan 2018 11:10:53 GMT
Github user alpinegizmo commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5317#discussion_r162906372
  
    --- Diff: docs/ops/config.md ---
    @@ -290,6 +290,12 @@ The following parameters configure Flink's JobManager and TaskManagers.
     
     - `taskmanager.network.numberOfBuffers` (deprecated, replaced by the three parameters
above): The number of buffers available to the network stack. This number determines how many
streaming data exchange channels a TaskManager can have at the same time and how well buffered
the channels are. If a job is rejected or you get a warning that the system has not enough
buffers available, increase this value (DEFAULT: **2048**). If set, it will be mapped to `taskmanager.network.memory.min`
and `taskmanager.network.memory.max` based on `taskmanager.memory.segment-size`.
     
    +- `taskmanager.network.memory.buffers-per-channel`: Number of network buffers to use
for each outgoing/incoming channel (subpartition/input channel). Especially in credit-based
flow control mode, it indicates how many credits are exclusive in each input channel. It should
be configured at least 2 for good performance. 1 buffer is for receving in-flight data in
the subpartition and 1 buffer is for parallel serialization. 
    +
    +- `taskmanager.network.memory.floating-buffers-per-gate`: Number of extra network buffers
to use for each outgoing/incoming gate (result partition/input gate). In credit-based flow
control mode, it indicates how many floating credits are shared for all the input channels.
The floating buffers are distributed based on backlog (real-time output buffers in the subpartition)
feedback. So the floating buffers can help relief back-pressure caused by imbalance data distribution
among subpartitions.
    +
    --- End diff --
    
    Number of extra network buffers used by each outgoing/incoming gate (result partition/input
gate). In credit-based flow control mode, this indicates how many floating credits are shared
among all the input channels. The floating buffers are distributed based on backlog (real-time
output buffers in the subpartition) feedback, and can help relieve back-pressure caused by
unbalanced data distribution among the subpartitions.


---

Mime
View raw message