flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ufuk Celebi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-5601) Window operator does not checkpoint watermarks
Date Sat, 21 Jan 2017 16:42:26 GMT
Ufuk Celebi created FLINK-5601:
----------------------------------

             Summary: Window operator does not checkpoint watermarks
                 Key: FLINK-5601
                 URL: https://issues.apache.org/jira/browse/FLINK-5601
             Project: Flink
          Issue Type: Improvement
          Components: State Backends, Checkpointing
            Reporter: Ufuk Celebi


During release testing [~stefanrichter83@gmail.com] and I noticed that watermarks are not
checkpointed in the window operator.

This can lead to non determinism when restoring checkpoints. I was running an adjusted {{SessionWindowITCase}}
via Kafka for testing migration and rescaling and ran into failures, because the data generator
required determinisitic behaviour.

What happened was that on restore it could happen that late elements were not dropped, because
the watermarks needed to be re-established after restore first.

[~aljoscha] Do you know whether there is a special reason for explicitly not checkpointing
watermarks?





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message