flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (FLINK-11917) Allow state schema migration from Kryo to POJO / Avro
Date Thu, 28 Mar 2019 08:41:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-11917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Tzu-Li (Gordon) Tai closed FLINK-11917.
    Resolution: Won't Fix

Subsumed by FLINK-12045. Closing.

> Allow state schema migration from Kryo to POJO / Avro
> -----------------------------------------------------
>                 Key: FLINK-11917
>                 URL: https://issues.apache.org/jira/browse/FLINK-11917
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / Type Serialization System
>            Reporter: Tzu-Li (Gordon) Tai
>            Priority: Major
> In Flink, it has been commonly advertised that users should try to avoid Kryo for state
serialization since it doesn't work out-of-the-box well for schema evolution stories. Kryo,
in the first place, wasn't designed with that in mind.
> In light of this, Flink should provide a migration path for state that were default to
be serialized by the {{KryoSerializer}} to other serializers that now support better schema
evolution capabilities, such as {{PojoSerializer}} and {{AvroSerializer}}.
> Essentially, what this means is that in the {{KryoSerializerSnapshot}} class's {{resolveSchemaCompatibility}}
method, we identify if the new serializer is either {{PojoSerializer}} or {{AvroSerializer}};
if so, we return {{TypeSerializerSchemaCompatibility.compatibleAfterMigration()}} as the result.
> For the user, this would allow them to simply upgrade their state types to be Avro-generated
{{SpecificRecord}} or a qualified POJO.

This message was sent by Atlassian JIRA

View raw message