flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhu Zhu (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-17726) Scheduler should take care of tasks directly canceled by TaskManager
Date Tue, 02 Jun 2020 02:57:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121514#comment-17121514
] 

Zhu Zhu commented on FLINK-17726:
---------------------------------

[~nicholasjiang] What I mean is that not all kinds of directly CANCELED tasks should trigger
failovers. 
There might be directly CANCELED tasks that were not caused by FAILED/CANCELED upstream tasks.
We still need to trigger failovers on this kind of directly CANCELED tasks, otherwise they
would stay in CANCELED forever.
A dedicate exception means the exception would only be thrown in this specific case. It does
not affect error handling of other cases.

> Scheduler should take care of tasks directly canceled by TaskManager
> --------------------------------------------------------------------
>
>                 Key: FLINK-17726
>                 URL: https://issues.apache.org/jira/browse/FLINK-17726
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination
>    Affects Versions: 1.11.0, 1.12.0
>            Reporter: Zhu Zhu
>            Assignee: Nicholas Jiang
>            Priority: Critical
>             Fix For: 1.11.0, 1.12.0
>
>
> JobManager will not trigger failure handling when receiving CANCELED task update. 
> This is because CANCELED tasks are usually caused by another FAILED task. These CANCELED
tasks will be restarted by the failover process triggered  FAILED task.
> However, if a task is directly CANCELED by TaskManager due to its own runtime issue,
the task will not be recovered by JM and thus the job would hang.
> This is a potential issue and we should avoid it.
> A possible solution is to let JobManager treat tasks transitioning to CANCELED from all
states except from CANCELING as failed tasks. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message