flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2897) Use distinct initial indices for OutputEmitter round-robin
Date Fri, 23 Oct 2015 10:04:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970756#comment-14970756

ASF GitHub Bot commented on FLINK-2897:

Github user fhueske commented on a diff in the pull request:

    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/operators/shipping/OutputEmitter.java
    @@ -143,14 +148,25 @@ public OutputEmitter(ShipStrategyType strategy, TypeComparator<T>
comparator, Pa
     	// --------------------------------------------------------------------------------------------
    +	private int[] forward() {
    +		if (this.channels == null) {
    --- End diff --
    This method preserves the behavior of `FORWARD` only if `numberOfChannels` is always 0
when `FORWARD` is configured which is out of control of the `OutputEmitter`.
    Should we add a check somewhere (e.g., in this condition to reduce the overhead) to ensure
that `numberOfChannels` is acutally 0 for `FORWARD`?

> Use distinct initial indices for OutputEmitter round-robin
> ----------------------------------------------------------
>                 Key: FLINK-2897
>                 URL: https://issues.apache.org/jira/browse/FLINK-2897
>             Project: Flink
>          Issue Type: Improvement
>          Components: Distributed Runtime
>    Affects Versions: 0.10
>            Reporter: Greg Hogan
>            Assignee: Greg Hogan
> Currently, when performing a round-robin partitioning each task will sequentially partition
starting with partition "1". This is fine in the usual case where the number of partitioned
objects greatly exceeds the number of channels. However, in the case where the number of objects
is relatively few (each, perhaps, requiring a large computation or access to an external system)
it would be much better to begin partitioning at distinct indices (the task index).

This message was sent by Atlassian JIRA

View raw message