flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leonard Wolters <leon...@sagent.io>
Subject Consistent (hashing) keyBy over multiple time or streaming windows
Date Mon, 02 Nov 2015 08:15:10 GMT

I was wondering if Flink already has implemented some sort of consistent 
keyBy mapping over multiple windows.
The underlying idea is to 'sessionize' incoming events over time (i.e. 
multiple streaming windows) on the same
partitions. As one can understand I want to avoid heavy shuffling over 
the network.

As far as I could read / understand from the API docs and the blogs on 
data-artisans, I can perfectly groupBy events
within one (time) window that are automatically distributed over all 
partitions. However, since sessions often exceed
the window submit duration, I want some guarantees that events belonging 
to the /same/ session are delivered to the
/same/ partition for new windows.

Is this possible or are they any plans to support this soon?

Thanks in advance,


Leonard Wolters
Chief Product Manager
*M*: +31 (0)6 55 53 04 01 | *T*: +31 (0)88 10 44 555
*E*: leonard@sagent.io <mailto:leonard@sagent.io> | *W*: sagent.io 
<http://sagent.io> | Disclaimer <http://sagent.io/email-disclaimer> | 
Sagent BV
Herengracht 504 | 1017CB Amsterdam | Netherlands

View raw message