flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2646) User functions should be able to differentiate between successful close and erroneous close
Date Tue, 28 Jan 2020 13:05:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17025092#comment-17025092
] 

Stephan Ewen commented on FLINK-2646:
-------------------------------------

While separating concerns is often a good idea, I think these things here are actually connected
  - Source signal on clean shutdown.
  - Atomic committing of side effects.
  - Doing this in a way that generalizes across batch and streaming. For example, make sure
that "final commit" on all tasks works even if tasks are not running any more.
  - Possibly a new sink interface design.

> User functions should be able to differentiate between successful close and erroneous
close
> -------------------------------------------------------------------------------------------
>
>                 Key: FLINK-2646
>                 URL: https://issues.apache.org/jira/browse/FLINK-2646
>             Project: Flink
>          Issue Type: Improvement
>          Components: API / DataStream
>    Affects Versions: 0.10.0
>            Reporter: Stephan Ewen
>            Assignee: Kostas Kloudas
>            Priority: Major
>              Labels: usability
>
> Right now, the {{close()}} method of rich functions is invoked in case of proper completion,
and in case of canceling in case of error (to allow for cleanup).
> In certain cases, the user function needs to know why it is closed, whether the task
completed in a regular fashion, or was canceled/failed.
> I suggest to add a method {{closeAfterFailure()}} to the {{RichFunction}}. By default,
this method calls {{close()}}. The runtime is the changed to call {{close()}} as part of the
regular execution and {{closeAfterFailure()}} in case of an irregular exit.
> Because by default all cases call {{close()}} the change would not be API breaking.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message