flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Teena Kappen // BPRISE <teena.kap...@bprise.com>
Subject Efficiency with different approaches of aggregation in Flink
Date Thu, 19 Apr 2018 06:15:46 GMT
Hi,

If I have to aggregate a value in a stream of records, which one of the below approaches will
be the most/least efficient?


  1.  Using a Global Window to aggregate the value and emit the record when it reaches a particular
threshold value.
  2.  Using a FlatMap with a State Variable which gets updated with each incoming record and
emit the record when it reaches the threshold value.
  3.  Using a FlatMap to store the aggregated value in an in-memory DB like Redis and query
the value and update it with each incoming record, and emit the record when it reaches the
threshold value.

Please rate the three approaches according to their efficiency.

Regards,
Teena

Mime
View raw message