flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhijiang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-10662) Refactor the ChannelSelector interface for single selected channel
Date Fri, 09 Nov 2018 10:08:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681198#comment-16681198

zhijiang commented on FLINK-10662:

[~pnowojski], I am currently working on this. If we change to return single {{int}} for {{ChannelSelector#selectChannels,
there are some choices for special }}{{BroadcastPartitioner}} implementation.
 # Return any int such as -1 for BroadcastPartitioner, because this value would not really
be used in {{RecordWriter}} side. We can make it shortcut branch for {{BroadcastPartitioner}}
in {{RecordWriter}}. This way is easy to handle, but may seem a little hacky to return a
dummy value in {{selectChannels}} method.
 # Return a `tuple<boolean, int>` for {{ChannelSelector#selectChannels}}, and the first
boolean value indicate whether it is broadcast or not. If broadcast, we will ignore the second
selected channel index in tuple. I am wondering if it would bring additional overhead than
the first way above.
 # Define a high level interface called {{ChannelSelectorBase}} with no specific methods,
then {{ChannelSelector}} extends {{ChannelSelectorBase}} and {{BroadcastPartitioner}} implements
{{ChannelSelectorBase}} directly, because it does not need any methods to return selected
channels. In this way, we change the current class structure and also need adjust the API
in {{DataStream}} to reference {{ChannelSelectorBase}} instead of {{StreamPartitioner}}. It
would be more complex to handle all the related references.

Currently, I would prefer the first way although it seems a little hacky, or do you have other
better suggestions? :)

> Refactor the ChannelSelector interface for single selected channel
> ------------------------------------------------------------------
>                 Key: FLINK-10662
>                 URL: https://issues.apache.org/jira/browse/FLINK-10662
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Network
>    Affects Versions: 1.5.4, 1.6.1
>            Reporter: zhijiang
>            Assignee: zhijiang
>            Priority: Minor
> In the discussion of broadcast improvement, [~pnowojski] pointed out the issue of improving
the current channel selector.
> In {{ChannelSelector#selectChannels}}, it would return an array for selected channels.
But considering specific implementations, only {{BroadcastPartitioner}} would select all the
channels, and other implementations will select one channel. So we can simple this interface
to return single channel index for benefiting performance, and specialize the {{BroadcastPartitioner}}
in a more efficient way.

This message was sent by Atlassian JIRA

View raw message