flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gyula Fora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-12142) Add beforeSnapshot method to StreamOperator interface
Date Tue, 09 Apr 2019 09:29:00 GMT
Gyula Fora created FLINK-12142:

             Summary: Add beforeSnapshot method to StreamOperator interface
                 Key: FLINK-12142
                 URL: https://issues.apache.org/jira/browse/FLINK-12142
             Project: Flink
          Issue Type: New Feature
          Components: Runtime / Checkpointing
            Reporter: Gyula Fora

Right now, when a stream task receives a triggerCheckpoint call, it immediately forwards
the checkpoint barrier and performs the checkpoint operation under the checkpoint lock.

This mechanism doesn't allow the stream operations to implement any logic that should be performed
before the snapshot is triggered in order to ensure state consistency.

Some examples would be:
 * Flushing any state (or other kind of) cache that is implemented in the operator layer
 * Wait for pending operations (maybe with a different async execution model)

The method could for instance look simply like this in the StreamOperator interface:
 {{default void beforeSnapshot(long checkpointId, long timestamp) throws Exception {};}} 

This method should then be called before the barrier is sent downstream still outside the
checkpoint lock.

This message was sent by Atlassian JIRA

View raw message