flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ygor Allan de Fraga (Jira)" <j...@apache.org>
Subject [jira] [Created] (FLINK-22761) Cannot remove POJO fields
Date Mon, 24 May 2021 15:03:00 GMT
Ygor Allan de Fraga created FLINK-22761:
-------------------------------------------

             Summary: Cannot remove POJO fields
                 Key: FLINK-22761
                 URL: https://issues.apache.org/jira/browse/FLINK-22761
             Project: Flink
          Issue Type: Bug
          Components: API / Type Serialization System
    Affects Versions: 1.12.1
            Reporter: Ygor Allan de Fraga


I tested a schema evolution in a state using POJO and no problem was found when trying to
add a new field, it was executed just fine. This same field was removed from the POJO as it
was just a test, but the application could not restore the state due to an error.

 

Here is the error:
{code:java}
2021-05-24 13:05:31,958 WARN  org.apache.flink.runtime.taskmanager.Task           
        [] - Co-Flat Map -> Map (3/3)#464 (e0e6d41a18214eab0a1d3c089d8672de) switched
from RUNNING to FAILED.2021-05-24 13:05:31,958 WARN  org.apache.flink.runtime.taskmanager.Task 
                  [] - Co-Flat Map -> Map (3/3)#464 (e0e6d41a18214eab0a1d3c089d8672de)
switched from RUNNING to FAILED.java.lang.Exception: Exception while creating StreamOperatorStateContext.
at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:254)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:272)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:425)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:535)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:525)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:565)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755)
[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:570)
[zdata-flink-streams.jar:0.1] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]Caused
by: org.apache.flink.util.FlinkException: Could not restore keyed state backend for CoStreamFlatMap_b101f370952ea85c2104e98dd54bf7f9_(3/3)
from any of the 1 provided restore options. at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
~[zdata-flink-streams.jar:0.1] ... 9 moreCaused by: org.apache.flink.runtime.state.BackendBuildingException:
Caught unexpected exception. at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackendBuilder.build(RocksDBKeyedStateBackendBuilder.java:361)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:587)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:93)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:328)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
~[zdata-flink-streams.jar:0.1] ... 9 moreCaused by: java.lang.NullPointerException at org.apache.flink.api.java.typeutils.runtime.PojoSerializer.<init>(PojoSerializer.java:119)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.api.java.typeutils.runtime.PojoSerializer.duplicate(PojoSerializer.java:184)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.api.java.typeutils.runtime.PojoSerializer.duplicate(PojoSerializer.java:56)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.api.java.typeutils.runtime.PojoSerializer.duplicate(PojoSerializer.java:170)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.api.java.typeutils.runtime.PojoSerializer.duplicate(PojoSerializer.java:56)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.api.common.typeutils.CompositeSerializer$PrecomputedParameters.precompute(CompositeSerializer.java:239)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.api.common.typeutils.CompositeSerializer.<init>(CompositeSerializer.java:52)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.state.ttl.TtlStateFactory$TtlSerializer.<init>(TtlStateFactory.java:270)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.state.ttl.TtlStateFactory$TtlSerializerSnapshot.createOuterSerializerWithNestedSerializers(TtlStateFactory.java:387)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.state.ttl.TtlStateFactory$TtlSerializerSnapshot.createOuterSerializerWithNestedSerializers(TtlStateFactory.java:354)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.restoreSerializer(CompositeTypeSerializerSnapshot.java:218)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.state.StateSerializerProvider.previousSchemaSerializer(StateSerializerProvider.java:186)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.state.StateSerializerProvider.currentSchemaSerializer(StateSerializerProvider.java:161)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.runtime.state.RegisteredKeyValueStateBackendMetaInfo.getStateSerializer(RegisteredKeyValueStateBackendMetaInfo.java:143)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.ttl.RocksDbTtlCompactFiltersManager.setAndRegisterCompactFilterIfStateTtl(RocksDbTtlCompactFiltersManager.java:73)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.RocksDBOperationUtils.createColumnFamilyDescriptor(RocksDBOperationUtils.java:167)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.RocksDBOperationUtils.createStateInfo(RocksDBOperationUtils.java:144)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.restore.AbstractRocksDBRestoreOperation.getOrRegisterStateColumnFamilyHandle(AbstractRocksDBRestoreOperation.java:178)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKVStateMetaData(RocksDBFullRestoreOperation.java:186)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKeyGroupsInStateHandle(RocksDBFullRestoreOperation.java:157)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restore(RocksDBFullRestoreOperation.java:142)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackendBuilder.build(RocksDBKeyedStateBackendBuilder.java:284)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:587)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:93)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:328)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345)
~[zdata-flink-streams.jar:0.1] at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163)
~[zdata-flink-streams.jar:0.1] ... 9 more{code}
 

A _RocksDb_ state backend is being used along with _valueState_ __.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message