flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrienne Kole <adrienneko...@gmail.com>
Subject AllWindowed vs Windowed with 1 key
Date Sun, 07 May 2017 21:23:45 GMT

I am doing simple aggregation with a keyed and global windows in flink.
When I compare the keyed window aggregation with 1 key and global window
(which has parallelism 1) I would expect that both of them would have
similar performance.

However, keyed stream with 1 key performs with 2x more throughput than
global window.
My configuration is 8 node cluster, 16 core in each node, parallelism = 128.

AFAIK, Flink doesn't manage skew by default and uses hash function to
assign keys to partitions. So if I have 1 key only, it should go to only
one partition always, which is semantically similar to global windows in

What can be the reason behind this difference in performance?


View raw message