flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhu Zhu (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-17726) Scheduler should take care of tasks directly canceled by TaskManager
Date Thu, 04 Jun 2020 06:43:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-17726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125588#comment-17125588
] 

Zhu Zhu commented on FLINK-17726:
---------------------------------

I think it should be that in TM we do not allow `transitionState` from a non-CANCELING state
to CANCELED.
e.g. `PartitionProducerStateResponseHandle#cancelConsumption()` should also be covered even
though it does not throw `CancelTaskException`.
However, given that there are still open questions about "how to consolidate root cause messages
which arrive out of order or late at the JobManager", so I think the design is not finalized.

> Scheduler should take care of tasks directly canceled by TaskManager
> --------------------------------------------------------------------
>
>                 Key: FLINK-17726
>                 URL: https://issues.apache.org/jira/browse/FLINK-17726
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination, Runtime / Task
>    Affects Versions: 1.11.0, 1.12.0
>            Reporter: Zhu Zhu
>            Assignee: Nicholas Jiang
>            Priority: Critical
>             Fix For: 1.12.0, 1.11.1
>
>
> JobManager will not trigger failure handling when receiving CANCELED task update. 
> This is because CANCELED tasks are usually caused by another FAILED task. These CANCELED
tasks will be restarted by the failover process triggered  FAILED task.
> However, if a task is directly CANCELED by TaskManager due to its own runtime issue,
the task will not be recovered by JM and thus the job would hang.
> This is a potential issue and we should avoid it.
> A possible solution is to let JobManager treat tasks transitioning to CANCELED from all
states except from CANCELING as failed tasks. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message