flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7378) Create a fix size (non rebalancing) buffer pool type for the floating buffers
Date Fri, 15 Sep 2017 07:16:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16167447#comment-16167447
] 

ASF GitHub Bot commented on FLINK-7378:
---------------------------------------

Github user zhijiangW commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4485#discussion_r139078792
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/buffer/NetworkBufferPool.java
---
    @@ -159,23 +159,20 @@ public void recycle(MemorySegment segment) {
     
     			this.numTotalRequiredBuffers += numRequiredBuffers;
     
    -			final List<MemorySegment> segments = new ArrayList<>(numRequiredBuffers);
    -			for (int i = 0 ; i < numRequiredBuffers ; i++) {
    -				segments.add(availableMemorySegments.poll());
    -			}
    +			redistributeBuffers();
    +		}
     
    +		final List<MemorySegment> segments = new ArrayList<>(numRequiredBuffers);
    +		for (int i = 0 ; i < numRequiredBuffers ; i++) {
     			try {
    -				redistributeBuffers();
    -			} catch (IOException e) {
    -				if (segments.size() > 0) {
    -					recycleMemorySegments(segments);
    -				}
    -
    +				segments.add(availableMemorySegments.take());
    --- End diff --
    
    Yes, I think this way seems better than always blocking to wait.  It is very nice of you
to give the specific test. 👍 


> Create a fix size (non rebalancing) buffer pool type for the floating buffers
> -----------------------------------------------------------------------------
>
>                 Key: FLINK-7378
>                 URL: https://issues.apache.org/jira/browse/FLINK-7378
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Core
>            Reporter: zhijiang
>            Assignee: zhijiang
>             Fix For: 1.4.0
>
>
> Currently the number of network buffers in {{LocalBufferPool}} for {{SingleInputGate}}
is limited by {{a * <number of channels> + b}}, where a is the number of exclusive buffers
for each channel and b is the number of floating buffers shared by all channels.
> Considering the credit-based flow control feature, we want to create a fix size buffer
pool used to manage the floating buffers for {{SingleInputGate}}. And the exclusive buffers
are assigned to {{InputChannel}}s directly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message