flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5041) Implement savepoint backwards compatibility 1.1 -> 1.2
Date Thu, 10 Nov 2016 13:35:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15654050#comment-15654050

ASF GitHub Bot commented on FLINK-5041:

GitHub user StefanRRichter opened a pull request:


    [FLINK-5041] Savepoint backwards compatibility 1.1 -> 1.2

    This PR implements sub-issues [FLINK-5042] and [FLINK-5043] of [FLINK-5041].
    [FLINK-5042] : Backwards compatibility for savepoints is implemented by reintroducing
some older classes from 1.1 that have been removed/replaced/changed in 1.2. All such classes
are moved to a different packet prefixed as `org.apache.flink.migration`, while their class
names remain unchanged. With `MigrationInstantiationUtil` a special deserialization facility
is provided that allows to load Java serialized objects of classes from 1.1. The idea is to
redirect their fully qualified classname to their new location in `org.apache.flink.migration`.
This is done through a modified `ObjectInputStream`. All state meta structures and handles
are then converted to 1.2.
    [FLINK-5043] : Introduced code in the keyed state backends ensures backwards compatibility
by translating states from the old backends based on Heap, Filesystem, and RocksDB.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StefanRRichter/flink savepoint-backwards-compatibility

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2781
commit a86f96df3c0be6e786246a6bbf814607fe7e0f47
Author: Stefan Richter <s.richter@data-artisans.com>
Date:   2016-11-01T11:29:01Z

    Backwards compatibility 1.1 -> 1.2

commit 49bc1fc10a47707d38f73fa9aa414f617439aef2
Author: Stefan Richter <s.richter@data-artisans.com>
Date:   2016-11-09T13:54:35Z

    Removing some unecessary code from migration classes

commit 9c578a37b11b5f70b92921d3562dd17ca9ed7493
Author: Stefan Richter <s.richter@data-artisans.com>
Date:   2016-11-09T17:21:13Z


commit 2791cbbfaeb828f99d1be6e7d460f17520389066
Author: Stefan Richter <s.richter@data-artisans.com>
Date:   2016-11-10T13:18:55Z

    Added migration unit test


> Implement savepoint backwards compatibility 1.1 -> 1.2
> ------------------------------------------------------
>                 Key: FLINK-5041
>                 URL: https://issues.apache.org/jira/browse/FLINK-5041
>             Project: Flink
>          Issue Type: New Feature
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.2.0
>            Reporter: Stefan Richter
>            Assignee: Stefan Richter
> This issue tracks the implementation of backwards compatibility between Flink 1.1 and
1.2 releases.
> This task subsumes:
> - Converting old savepoints to new savepoints, including a conversion of state handles
to their new replacement.
> - Converting keyed state from old backend implementations to their new counterparts.
> - Converting operator and function state for all changed operators.

This message was sent by Atlassian JIRA

View raw message