flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1967) Introduce (Event)time in Streaming
Date Fri, 03 Jul 2015 13:39:05 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14613190#comment-14613190

ASF GitHub Bot commented on FLINK-1967:

Github user aljoscha commented on a diff in the pull request:

    --- Diff: flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/streamrecord/StreamRecord.java
    @@ -17,87 +17,104 @@
     package org.apache.flink.streaming.runtime.streamrecord;
    -import java.io.Serializable;
    -import org.apache.flink.api.java.functions.KeySelector;
    -import org.apache.flink.api.java.tuple.Tuple;
    +import org.joda.time.Instant;
    - * Object for wrapping a tuple or other object with ID used for sending records
    - * between streaming task in Apache Flink stream processing.
    + * One value in a data stream. This stores the value and the associated timestamp.
    -public class StreamRecord<T> implements Serializable {
    -	private static final long serialVersionUID = 1L;
    +public class StreamRecord<T> {
    -	private T streamObject;
    -	public boolean isTuple;
    +	// We store it as Object so that we can reuse a StreamElement for emitting
    +	// elements of a different type while still reusing the timestamp.
    +	private Object value;
    --- End diff --
    But then it just requires casts in another place...

> Introduce (Event)time in Streaming
> ----------------------------------
>                 Key: FLINK-1967
>                 URL: https://issues.apache.org/jira/browse/FLINK-1967
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
> This requires introducing a timestamp in streaming record and a change in the sources
to add timestamps to records. This will also introduce punctuations (or low watermarks) to
allow windows to work correctly on unordered, timestamped input data. In the process of this,
the windowing subsystem also needs to be adapted to use the punctuations. Furthermore, all
operators need to be made aware of punctuations and correctly forward them. Then, a new operator
must be introduced to to allow modification of timestamps.

This message was sent by Atlassian JIRA

View raw message