flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3554) Bounded sources should emit a Max Watermark when they are done
Date Tue, 01 Mar 2016 13:38:18 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15173758#comment-15173758
] 

ASF GitHub Bot commented on FLINK-3554:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1750#discussion_r54566728
  
    --- Diff: flink-streaming-java/src/main/java/org/apache/flink/streaming/api/operators/StreamSource.java
---
    @@ -66,19 +71,60 @@ public void run(final Object lockingObject, final Output<StreamRecord<OUT>>
coll
     				throw new Exception(String.valueOf(timeCharacteristic));
     		}
     
    -		userFunction.run(ctx);
    -
    -		ctx.close();
    +		// copy to a field to give the 'cancel()' method access
    +		this.ctx = ctx;
    +		
    +		try {
    +			userFunction.run(ctx);
    +
    +			// if we get here, then the user function either exited after being done (finite source)
    +			// or the function was canceled or stopped. For the finite source case, we should
emit
    +			// a final watermark that indicates that we reached the end of event-time
    +			if (!isCanceledOrStopped()) {
    +				ctx.emitWatermark(Watermark.MAX_WATERMARK);
    +			}
    +		} finally {
    +			// make sure that the context is closed in any case
    +			ctx.close();
    +		}
     	}
     
     	public void cancel() {
    +		// important: marking the source as stopped has to happen before the function is stopped.
    +		// the flag that tracks this status is volatile, to the memory model also guarantees
    --- End diff --
    
    typo in comment: to => so


> Bounded sources should emit a Max Watermark when they are done
> --------------------------------------------------------------
>
>                 Key: FLINK-3554
>                 URL: https://issues.apache.org/jira/browse/FLINK-3554
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>    Affects Versions: 1.0.0
>            Reporter: Stephan Ewen
>            Priority: Critical
>             Fix For: 1.0.1
>
>
> For proper event time support in bounded sources, these sources should emit a final watermark
before shutting down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message