flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi (JIRA) <j...@apache.org>
Subject [jira] [Commented] (FLINK-2221) Checkpoints to "file://" are not cleaned up
Date Mon, 15 Jun 2015 14:08:02 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586078#comment-14586078
] 

Márton Balassi commented on FLINK-2221:
---------------------------------------

So the current architecture has the assumption that the JobManager has access to the checkpoint
files the same way a TaskManager does. We should make this more clear for the user.

An additional fragility factor can be that the file state provider simply adds a random string
to the directory name and expects it to be an unused name, or simply overwrites it. It is
highly unlikely that this behaviur would cause a trouble, but let us still avoid it. Let me
add a quick fix for the latter issue.

> Checkpoints to "file://" are not cleaned up
> -------------------------------------------
>
>                 Key: FLINK-2221
>                 URL: https://issues.apache.org/jira/browse/FLINK-2221
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>            Reporter: Aljoscha Krettek
>
> If you think about it, this could never work. The state handle cleanup logic happens
purely on the JobManager. So what happens is that the TaskManagers create state in some directory,
let's say /tmp/checkpoints, on the TaskManager. For cleanup, the JobManager gets the state
handle and calls discard (on the JobManager), this tries to cleanup the state in /tmp/checkpoints,
but of course, there is nothing there since we are still on the JobManager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message