flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Flink Jira Bot (Jira)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-15325) Input location preference which affects task distribution may make certain job performance worse
Date Mon, 24 May 2021 22:51:03 GMT

     [ https://issues.apache.org/jira/browse/FLINK-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Flink Jira Bot updated FLINK-15325:
-----------------------------------
      Labels: auto-deprioritized-major  (was: stale-major)
    Priority: Minor  (was: Major)

This issue was labeled "stale-major" 7 ago and has not received any updates so it is being
deprioritized. If this ticket is actually Major, please raise the priority and ask a committer
to assign you the issue or revive the public discussion.


> Input location preference which affects task distribution may make certain job performance
worse 
> -------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-15325
>                 URL: https://issues.apache.org/jira/browse/FLINK-15325
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / Coordination
>    Affects Versions: 1.10.0
>            Reporter: Zhu Zhu
>            Priority: Minor
>              Labels: auto-deprioritized-major
>         Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png
>
>
> When running TPC-DS jobs in a session cluster, we observed that sometimes tasks are not
evenly distributed in TMs. The root cause turned out to be that the downstream tasks tend
to be TM or host local with its input tasks. This helps to reduce network shuffle. 
> However, in certain cases, like the topology presented in the attached image, jamming
the input task's TM and machine with downstream tasks would affect the performance. In this
case, respecting input location preferences is causing troubles more than bringing benefits.
> So I'm wondering whether we should introduce a config so that users can disable input
location preferences?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message