flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5768) Apply new aggregation functions for datastream and dataset tables
Date Wed, 01 Mar 2017 01:34:45 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889278#comment-15889278
] 

ASF GitHub Bot commented on FLINK-5768:
---------------------------------------

Github user shaoxuan-wang commented on the issue:

    https://github.com/apache/flink/pull/3423
  
    @fhueske  thanks for the review. I completely agree with your suggestion on  "reworking
the batch design". Actually I have proposed the same idea to "rework the batch" before your
review (maybe you have missed my comment on Feb.27). I was hesitating to make the changes,
as I want to keep this PR as dedicated as possible. But since the performance of the current
design is a concern, let's do the clean up all together within this PR.  Regarding to "pairwise
merging", I have different opinion, there are many aggregates that merging a list at once
is much more efficient than merge just two. If we always use "pairwise merging" in the runtime,
we will lose the advantage of merge(List) API defined in AggregateFunction interface. If the
memory is a concern, we can limit the size of List for each merge iteration, I will provide
the update very soon.


> Apply new aggregation functions for datastream and dataset tables
> -----------------------------------------------------------------
>
>                 Key: FLINK-5768
>                 URL: https://issues.apache.org/jira/browse/FLINK-5768
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table API & SQL
>            Reporter: Shaoxuan Wang
>            Assignee: Shaoxuan Wang
>
> Apply new aggregation functions for datastream and dataset tables
> This includes:
> 1. Change the implementation of the DataStream aggregation runtime code to use new aggregation
functions and aggregate dataStream API.
> 2. DataStream will be always running in incremental mode, as explained in 06/Feb/2017
in FLINK5564.
> 2. Change the implementation of the Dataset aggregation runtime code to use new aggregation
functions.
> 3. Clean up unused class and method.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message