flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kostas Kloudas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8522) DefaultOperatorStateBackend writes data in checkpoint that is never read.
Date Tue, 30 Jan 2018 09:48:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344764#comment-16344764

Kostas Kloudas commented on FLINK-8522:

@[~tzulitai] : I think that if the person that extends the code in future versions by adding
data after the MetaInfo, also implements the migration policy correctly (i.e. bump the VERSION),
then we will have no problem, as when he restores from the savepoint, the correct `restore()`
method will be called, and it will not try to read the newly added data.

Given this, I will go on and simply remove the line.

> DefaultOperatorStateBackend writes data in checkpoint that is never read.
> -------------------------------------------------------------------------
>                 Key: FLINK-8522
>                 URL: https://issues.apache.org/jira/browse/FLINK-8522
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.4.0
>            Reporter: Kostas Kloudas
>            Assignee: Kostas Kloudas
>            Priority: Major
>             Fix For: 1.4.1
> In the DefaultOperatorStateBackend at line 283 we write in the checkpoint an int declaring
the number of the operator states that we include in the checkpoint. 
> This number is never read when restoring and this can lead to confusion and problems
with backwards compatibility and/or extension of the types of state we support (e.g. broadcast
> There are two easy solutions, either remove the line and do not write the size, or make
sure that we also read this number when restoring and simply ignore it.
> I would go for the first one, i.e. remove the line. What do you think [~richtesn] and
[~tzulitai] ?

This message was sent by Atlassian JIRA

View raw message