flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-10977) Add FlatAggregate operator to unbounded streaming Table API
Date Wed, 12 Dec 2018 11:44:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16718856#comment-16718856
] 

ASF GitHub Bot commented on FLINK-10977:
----------------------------------------

dianfu commented on a change in pull request #7209: [FLINK-10977][table] Add UnBounded FlatAggregate
operator to streaming Table API
URL: https://github.com/apache/flink/pull/7209#discussion_r240945195
 
 

 ##########
 File path: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/api/table.scala
 ##########
 @@ -1028,6 +1061,119 @@ class Table(
   }
 }
 
+class FlatAggTable(
+  private[flink] val table: Table,
+  private[flink] val tableAggCall: TableAggFunctionCall) {
+
+  /**
+    * Performs a selection operation on a FlatAggTable table. Similar to an SQL SELECT statement.
+    * The field expressions can contain complex expressions.
+    *
+    * __Note__: You have to close the flatAggregate with a select statement. And the select
+    * statement does not support * and aggregate functions.
+    *
+    * Example:
+    *
+    * {{{
+    *   val tableAggFunc: TableAggregateFunction = new MyTableAggregateFunction
+    *   tab.flatAggregate(tableAggFunc('a, 'b)).select(udf('_1), '_2, '_3)
+    * }}}
+    */
+  def select(fields: Expression*): Table = {
+    new GroupedFlatAggTable(table, Nil, tableAggCall).select(fields: _*)
+  }
+
+  /**
+    * Performs a selection operation on a FlatAggTable table. Similar to an SQL SELECT statement.
+    * The field expressions can contain complex expressions.
+    *
+    * __Note__: You have to close the flatAggregate with a select statement. And the select
+    * statement does not support * and aggregate functions.
+    *
+    * Example:
+    *
+    * {{{
+    *   val tableAggFunc: TableAggregateFunction = new MyTableAggregateFunction
+    *   tableEnv.registerFunction("myTableAggFunc", tableAggFunc);
+    *   tab.flatAggregate("myTableAggFunc(a, b)").select("_1, _2, _3")
+    * }}}
+    */
+  def select(fields: String): Table = {
+    new GroupedFlatAggTable(table, Nil, tableAggCall).select(fields)
+  }
+}
+
+/**
+  * A table that has been grouped on a set of grouping keys and perform flatAggregate.
+  */
+class GroupedFlatAggTable(
+  private[flink] val table: Table,
+  private[flink] val groupKey: Seq[Expression],
+  private[flink] val tableAggCall: TableAggFunctionCall) {
+
+  /**
+    * Performs a selection operation on a GroupedFlatAggTable table. Similar to an SQL SELECT
+    * statement. The field expressions can contain complex expressions.
+    *
+    * __Note__: You have to close the flatAggregate with a select statement. And the select
+    * statement does not support * and aggregate functions.
+    *
+    * Example:
+    *
+    * {{{
+    *   val tableAggFunc: TableAggregateFunction = new MyTableAggregateFunction
+    *   tab.groupBy('key).flatAggregate(tableAggFunc('a, 'b)).select(udf('_1), '_2, '_3)
+    * }}}
+    */
+  def select(fields: Expression*): Table = {
+    val expandedFields = expandProjectList(tableAggCall.args, table.logicalPlan, table.tableEnv)
+    val projectFields = extractFieldReferences(expandedFields ++ groupKey)
+
+    val flatAggTable = new Table(table.tableEnv,
+      TableAggregate(groupKey, tableAggCall,
+        Project(projectFields, table.logicalPlan).validate(table.tableEnv)
+      ).validate(table.tableEnv))
+
+    // check no '*' in the select of flatAggregate.
 
 Review comment:
   Why * is not supported? I think * should not be supported at present in group window case.
Right?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Add FlatAggregate operator to unbounded streaming Table API
> -----------------------------------------------------------
>
>                 Key: FLINK-10977
>                 URL: https://issues.apache.org/jira/browse/FLINK-10977
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table API &amp; SQL
>            Reporter: sunjincheng
>            Assignee: Hequn Cheng
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.8.0
>
>
> Add FlatAggregate operator to streaming Table API as described in [Google doc|https://docs.google.com/document/d/1tnpxg31EQz2-MEzSotwFzqatsB4rNLz0I-l_vPa5H4Q/edit#heading=h.q23rny2iglsr].
> The usage:
> {code:java}
> val res = tab
> .groupBy('a) // leave out groupBy-clause to define global table aggregates
> .flatAgg(fun: TableAggregateFunction) // output has columns 'a, 'b, 'c
> .select('a, 'c){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message