flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3428) Add fixed time trailing timestamp/watermark extractor
Date Thu, 28 Apr 2016 11:28:12 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261999#comment-15261999

ASF GitHub Bot commented on FLINK-3428:

Github user rmetzger commented on a diff in the pull request:

    --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/AscendingTimestampExtractor.java
    @@ -20,123 +20,18 @@
     import org.apache.flink.annotation.PublicEvolving;
    -import org.apache.flink.streaming.api.watermark.Watermark;
    -import org.slf4j.Logger;
    -import org.slf4j.LoggerFactory;
    -import static java.util.Objects.requireNonNull;
      * A timestamp assigner and watermark generator for streams where timestamps are monotonously
      * ascending. In this case, the local watermarks for the streams are easy to generate,
      * they strictly follow the timestamps.
    + * <b>Note:</b> This is just a stub class. The actual code for this has moved
    + * {@link org.apache.flink.streaming.api.functions.timestamps.AscendingTimestampExtractor}.
    + *
    --- End diff --
    What's the reasoning behind this stub class?

> Add fixed time trailing timestamp/watermark extractor
> -----------------------------------------------------
>                 Key: FLINK-3428
>                 URL: https://issues.apache.org/jira/browse/FLINK-3428
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Robert Metzger
>            Assignee: Kostas Kloudas
> Flink currently provides only one build-in timestamp extractor, which assumes strictly
ascending timestamps. In real world use cases, timestamps are almost never strictly ascending.
> Therefore, I propose to provide an utility watermark extractor which is generating watermarks
with a fixed-time trailing.
> The implementation should keep track of the highest event-time seen so far and subtract
a fixed amount of time from that event time.
> This way, users can for example specify that the watermarks should always "lag behind"
10 minutes.

This message was sent by Atlassian JIRA

View raw message