flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi (JIRA) <j...@apache.org>
Subject [jira] [Commented] (FLINK-2221) Checkpoints to "file://" are not cleaned up
Date Mon, 15 Jun 2015 14:48:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586135#comment-14586135
] 

Márton Balassi commented on FLINK-2221:
---------------------------------------

We can go down the check and retry route, but as it is distributed systems that is still not
safe. My current approach is to add some prefix consisting of jobname, datetime, operatorname
and parallel instance name that makes it more unique. Or we could also generate a UUID.

> Checkpoints to "file://" are not cleaned up
> -------------------------------------------
>
>                 Key: FLINK-2221
>                 URL: https://issues.apache.org/jira/browse/FLINK-2221
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>            Reporter: Aljoscha Krettek
>
> If you think about it, this could never work. The state handle cleanup logic happens
purely on the JobManager. So what happens is that the TaskManagers create state in some directory,
let's say /tmp/checkpoints, on the TaskManager. For cleanup, the JobManager gets the state
handle and calls discard (on the JobManager), this tries to cleanup the state in /tmp/checkpoints,
but of course, there is nothing there since we are still on the JobManager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message