flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Access Sliding window
Date Fri, 04 Aug 2017 19:49:49 GMT
Hi Raj,

you have to combine two streams. The first stream has the running avg +
std-dev over the last 6 hours, the second stream has the 15 minute counts.
Both streams emit one record every 15 minutes. What you wan to do is to
join the two records of both streams with the same timestamp.
You do that by connecting the streams and implementing a CoProcessFunction.
btw. CoProcessFunction implements RichFunction.
The function must be stateful, because you need to collect the first record
that is received from either input and wait for the record from the other
input in order to be able to join them, i.e. compare the count against the
avg + std-dev. So which ever record you receive first, you put into state
and wait for the other record with the same timestamp to arrive. After the
join, you clear the state.

Hope that helps,

2017-08-04 7:49 GMT+02:00 Raj Kumar <smallthings1992@gmail.com>:

> Thanks Fabian. Your suggestion helped. But, I am stuck at 3rd step
> 1. I didn't completely understand the step 3. What the process function
> should look like ? Why does it needs to be stateful. Can you please provide
> more details on this.
> 2. In the stateful, function, we need to have a value state ? what details
> we need to store would be helpful to implement the use case.
> 3. Moreover, I see that RichProcessFunction is deprecated. What else can we
> use in place of RichProcessFunction ?
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Access-Sliding-
> window-tp14519p14675.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.

View raw message