flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ufuk Celebi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2929) Recovery of jobs on cluster restarts
Date Tue, 27 Oct 2015 17:47:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14976835#comment-14976835

Ufuk Celebi commented on FLINK-2929:

Maybe you are right. Keeping it as it is (and adding an option to purge) makes sure that jobs
are not removed accidentally. And it's possible to cancel an old job after a restart. 

> Recovery of jobs on cluster restarts
> ------------------------------------
>                 Key: FLINK-2929
>                 URL: https://issues.apache.org/jira/browse/FLINK-2929
>             Project: Flink
>          Issue Type: Improvement
>    Affects Versions: 0.10
>            Reporter: Ufuk Celebi
> Recovery information is stored in ZooKeeper under a static root like {{/flink}}. In case
of a cluster restart without canceling running jobs old jobs will be recovered from ZooKeeper.
> This can be confusing or helpful depending on the use case.
> I suspect that the confusing case will be more common.
> We can change the default cluster start up (e.g. new YARN session or new ./start-cluster
call) to purge all existing data in ZooKeeper and add a flag to not do this if needed.
> [~trohrmann@apache.org], [~aljoscha], [~StephanEwen] what's your opinion?

This message was sent by Atlassian JIRA

View raw message