flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Smirnov <alexander.smirn...@gmail.com>
Subject Checkpointing barriers
Date Mon, 23 Apr 2018 14:21:56 GMT
Hi,

I'm reading documentation about checkpointing:
https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html
It describes a case, when an operator receives data from all its incoming
streams alongs with barriers. There's also an illustration on that page for
the case.

One thing confuses me, though.

Each data stream has its own source which emits own sequence of barriers.
This is very implementation specific, and the docs say that for incoming
Kafka streams the connector uses offset information to produce a barrier.
But the illustration and the explanation have "barrier n" in both of the
incoming streams.

"As soon as the operator receives snapshot barrier n from an incoming
stream, it cannot process any further records from that stream until it has
received the barrier n from the other inputs as well"

Can you please help me to understand why "barrier n" would appear in
different streams.
I'd expect "barrier n" in stream1 and "barrier m" in stream2

Thank you,
Alex

Mime
View raw message