flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9034) State Descriptors drop TypeInformation on serialization
Date Wed, 21 Mar 2018 20:17:01 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16408519#comment-16408519
] 

ASF GitHub Bot commented on FLINK-9034:
---------------------------------------

Github user aljoscha commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5732#discussion_r176222855
  
    --- Diff: flink-core/src/test/java/org/apache/flink/api/common/state/StateDescriptorTest.java
---
    @@ -130,6 +135,31 @@ public void testInitializeSerializerAfterSerializationWithCustomConfig()
throws
     				.getRegistration(File.class).getId() > 0);
     	}
     
    +	// ------------------------------------------------------------------------
    +	//  Tests for serializer initialization
    +	// ------------------------------------------------------------------------
    +
    +	/**
    +	 * FLINK-6775, tests that the returned serializer is duplicated.
    +	 * This allows to share the state descriptor across threads.
    +	 */
    +	@Test
    +	public void testSerializerDuplication() throws Exception {
    +		// we need a serializer that actually duplicates for testing (a stateful one)
    --- End diff --
    
    Same as above, we should assert that assumption


> State Descriptors drop TypeInformation on serialization
> -------------------------------------------------------
>
>                 Key: FLINK-9034
>                 URL: https://issues.apache.org/jira/browse/FLINK-9034
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.5.0, 1.4.2
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>            Priority: Major
>             Fix For: 1.6.0
>
>
> The following code currently causes problems
> {code}
> public class MyFunction extends RichMapFunction<A, B>  {
>     private final ValueStateDescriptor<MyType> descr = new ValueStateDescriptor<>("state
name", MyType.class);
>     private ValueState<MyType> state;
>     @Override
>     public void open() {
>         state = getRuntimeContext().getValueState(descr);
>     }
> }
> {code}
> The problem is that the state descriptor drops the type information and creates a serializer
before serialization as part of shipping the function in the cluster. To do that, it initializes
the serializer with an empty execution config, making serialization inconsistent.
> This is mainly an artifact from the days when dropping the type information before shipping
was necessary, because the type info was not serializable. It now is, and we can fix that
bug.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message