flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2301) In BarrierBuffer newer Barriers trigger old Checkpoints
Date Wed, 01 Jul 2015 08:24:05 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609727#comment-14609727
] 

Stephan Ewen commented on FLINK-2301:
-------------------------------------

I agree that we should change that behavior.

With the current guarantees and recovery, we should never lose a barrier, so we are not running
into an issue there (as far as I know).

But it is always good design that each component in itself behaves consistently, which would
mean that the barrier buffer either throws an exception in case of receiving unrelated barriers,
or drop the previous barriers that were overtaken on at leas one input.

> In BarrierBuffer newer Barriers trigger old Checkpoints
> -------------------------------------------------------
>
>                 Key: FLINK-2301
>                 URL: https://issues.apache.org/jira/browse/FLINK-2301
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
>
> When the BarrierBuffer has some inputs blocked on barrier 0, then receives barriers for
barrier 1 on the other inputs this makes the BarrierBuffer process the checkpoint with id
0.
> I think the BarrierBuffer should drop all previous BarrierCheckpoints when it receives
a barrier from a more recent checkpoint and unblock the previously blocked channels. This
will make it ready to correctly react to the other barriers of the newer checkpoint. It should
also ignore barriers that arrive late when we already processed a more recent checkpoint.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message