flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9034) State Descriptors drop TypeInformation on serialization
Date Thu, 22 Mar 2018 08:49:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16409230#comment-16409230

ASF GitHub Bot commented on FLINK-9034:

Github user StephanEwen commented on the issue:

    Concerning guarding the test assumptions: I think it is fine, because if the KryoSerializer
is not duplicating properly any more (assumption violated), the test will also fail. (The
problematic case would be the other way around, a serializer that does not duplicate and we
check that the same instance is returned).
    @aljoscha Do you think this should go into 1.5 as well?
    @aljoscha Concerning changing `initializeSerializerUnlessSet(ExecutionConfig)` to `getSerializer(ExecutionConfig)`
- I think that is a good idea, but would do that in a separate step. We need to re-work anyways
a bit how we store state descriptors and obtain and reconfigure serializers.

> State Descriptors drop TypeInformation on serialization
> -------------------------------------------------------
>                 Key: FLINK-9034
>                 URL: https://issues.apache.org/jira/browse/FLINK-9034
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.5.0, 1.4.2
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Major
>             Fix For: 1.6.0
> The following code currently causes problems
> {code}
> public class MyFunction extends RichMapFunction<A, B>  {
>     private final ValueStateDescriptor<MyType> descr = new ValueStateDescriptor<>("state
name", MyType.class);
>     private ValueState<MyType> state;
>     @Override
>     public void open() {
>         state = getRuntimeContext().getValueState(descr);
>     }
> }
> {code}
> The problem is that the state descriptor drops the type information and creates a serializer
before serialization as part of shipping the function in the cluster. To do that, it initializes
the serializer with an empty execution config, making serialization inconsistent.
> This is mainly an artifact from the days when dropping the type information before shipping
was necessary, because the type info was not serializable. It now is, and we can fix that

This message was sent by Atlassian JIRA

View raw message