flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8581) Improve performance for low latency network
Date Mon, 08 Oct 2018 12:41:02 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641771#comment-16641771
] 

ASF GitHub Bot commented on FLINK-8581:
---------------------------------------

NicoK commented on a change in pull request #6698: [FLINK-8581][network] Move flushing remote
subpartitions from OutputFlusher to netty
URL: https://github.com/apache/flink/pull/6698#discussion_r223340950
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/io/network/partition/ResultPartition.java
 ##########
 @@ -94,6 +96,19 @@
 	/** The subpartitions of this partition. At least one. */
 	private final ResultSubpartition[] subpartitions;
 
+	/**
+	 * Subset of {@code subpartitions} that are definitely local. We can only determine whether
a
+	 * subpartition is local or not once it's read view was created.
+	 */
+	private final ArrayList<ResultSubpartition> localSubpartitions = new ArrayList<>();
+
+	/**
+	 * Subset of {@code subpartitions} that are definitely remote, however once we determined
that,
+	 * we haven't yet known about {@link #flushTimeout}. This has to be handled during
+	 * {@link #setFlushTimeout(long)}.
+	 */
+	private final ArrayList<ResultSubpartition> remoteSubpartitionsMissingPeriodicFlushes
= new ArrayList<>();
+
 
 Review comment:
   actually, how about the following idea that should make things a bit simpler / improve
the abstraction:
   - only differentiate between self-flushing channels (periodically, after registration)
and channels that require manual flushing (instead of remove vs. local)
   - have `PipelinedSubpartition#registerPeriodicFlush()` return whether registering for "self-flush"
worked or not
   - have `PipelinedSubpartition#registerPeriodicFlush()` dealing with the local vs. non-local
nature of the channel
   - only have `ResultSubpartition[] subpartitions` and `ArrayList<ResultSubpartition>
manuallyFlushedSubpartitions` members

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Improve performance for low latency network
> -------------------------------------------
>
>                 Key: FLINK-8581
>                 URL: https://issues.apache.org/jira/browse/FLINK-8581
>             Project: Flink
>          Issue Type: Improvement
>          Components: Network
>    Affects Versions: 1.5.0
>            Reporter: Piotr Nowojski
>            Assignee: Piotr Nowojski
>            Priority: Major
>              Labels: pull-request-available
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message