flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aljoscha <...@git.apache.org>
Subject [GitHub] flink pull request: Enhance Partitioned State and use it in Window...
Date Tue, 01 Mar 2016 10:15:27 GMT
Github user aljoscha commented on the pull request:

    https://github.com/apache/flink/pull/1562#issuecomment-190649095
  
    Hi @wenlonglwl, the reason for this were mostly practical concerns. We want to use our
partitioned state abstraction for the window state because that makes it easy to change the
state backend depending on the use case. So, when using a ReduceFunction this is actually
put into a ReducingState that keeps an aggregated value using that ReduceFunction. Having
RichFunctions in inside state would require a lot of overhead. First the function would have
to be copied for every different key because the function can keep state internally, second,
it would require wiring all the additional RichFunction calls (such as open(), close(), the
RuntimeContext) through to the state abstraction.
    
    Could you maybe use the `WindowedStream.apply(ReduceFunction, WindowFunction)` method,
this allows you to incrementally aggregate the elements in the window and then get the final
result in the WindowFunction once the window is emitted. This WindowFunction can be a RichFunction
and can also keep state.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message