flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3428) Add fixed time trailing timestamp/watermark extractor
Date Thu, 28 Apr 2016 11:28:12 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261999#comment-15261999
] 

ASF GitHub Bot commented on FLINK-3428:
---------------------------------------

Github user rmetzger commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1764#discussion_r61411342
  
    --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/AscendingTimestampExtractor.java
---
    @@ -20,123 +20,18 @@
     
     import org.apache.flink.annotation.PublicEvolving;
     
    -import org.apache.flink.streaming.api.watermark.Watermark;
    -import org.slf4j.Logger;
    -import org.slf4j.LoggerFactory;
    -
    -import static java.util.Objects.requireNonNull;
    -
     /**
      * A timestamp assigner and watermark generator for streams where timestamps are monotonously
      * ascending. In this case, the local watermarks for the streams are easy to generate,
because
      * they strictly follow the timestamps.
      *
    + * <b>Note:</b> This is just a stub class. The actual code for this has moved
to
    + * {@link org.apache.flink.streaming.api.functions.timestamps.AscendingTimestampExtractor}.
    + *
    --- End diff --
    
    What's the reasoning behind this stub class?


> Add fixed time trailing timestamp/watermark extractor
> -----------------------------------------------------
>
>                 Key: FLINK-3428
>                 URL: https://issues.apache.org/jira/browse/FLINK-3428
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Robert Metzger
>            Assignee: Kostas Kloudas
>
> Flink currently provides only one build-in timestamp extractor, which assumes strictly
ascending timestamps. In real world use cases, timestamps are almost never strictly ascending.
> Therefore, I propose to provide an utility watermark extractor which is generating watermarks
with a fixed-time trailing.
> The implementation should keep track of the highest event-time seen so far and subtract
a fixed amount of time from that event time.
> This way, users can for example specify that the watermarks should always "lag behind"
10 minutes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message