flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-18038) StateBackendLoader logs application-defined state before it is fully configured
Date Thu, 04 Jun 2020 01:25:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125454#comment-17125454

Yu Li commented on FLINK-18038:

+1 to log the final configured backend, while I think it still values to keep the log of the
original backend from application, in case something went wrong and we'd like to know which
part didn't work well. What do you think? [~sbairos]

Will assign the JIRA and review the PR once we got a consensus. Thanks.

> StateBackendLoader logs application-defined state before it is fully configured
> -------------------------------------------------------------------------------
>                 Key: FLINK-18038
>                 URL: https://issues.apache.org/jira/browse/FLINK-18038
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / State Backends
>    Affects Versions: 1.9.1
>            Reporter: Steve Bairos
>            Priority: Trivial
> In the [StateBackendLoader|[https://github.com/apache/flink/blob/bb46756b84940a6134910e74406bfaff4f2f37e9/flink-runtime/src/main/java/org/apache/flink/runtime/state/StateBackendLoader.java#L201]],
there's this log line:
> {code:java}
> logger.info("Using application-defined state backend: {}", fromApplication); {code}
> It seems like this is inaccurate though because immediately after logging this, if fromApplication
is a ConfigurableStateBackend, we call the .configure() function and it is replaced by a newly
configured StateBackend. 
> To me, it seems like it would be better if we logged the state backend after it was fully
configured. In the current setup, we get confusing logs like this: 
> {code:java}
> 2020-05-29 21:39:44,387 INFO  org.apache.flink.streaming.runtime.tasks.StreamTask   
       - Using application-defined state backend: RocksDBStateBackend{checkpointStreamBackend=File
State Backend (checkpoints: 's3://pinterest-montreal/checkpoints/xenon-dev-001-20191210/Xenon/BasicJavaStream',
savepoints: 'null', asynchronous: UNDEFINED, fileStateThreshold: -1), localRocksDbDirectories=null,
enableIncrementalCheckpointing=UNDEFINED, numberOfTransferingThreads=-1}2020-05-29 21:39:44,387
INFO  org.apache.flink.streaming.runtime.tasks.StreamTask           - Configuring application-defined
state backend with job/cluster config{code}
> Which makes it ambiguous whether or not settings in our flink-conf.yaml like "state.backend.incremental:
true" are being applied properly or not. 
> I can make a diff for the change if there aren't any objections

This message was sent by Atlassian Jira

View raw message