flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5624) Support tumbling window on streaming tables in the SQL API
Date Tue, 07 Feb 2017 17:28:41 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856362#comment-15856362
] 

Fabian Hueske commented on FLINK-5624:
--------------------------------------

Copying a comment from FLINK-5710.

The idea is to kind of apply pattern matching for certain expressions such as {{GROUP BY FLOOR(procTime()
TO HOURS)}}.
The challenge is that this expression is spread across several LogicalRel nodes.
The expression {{FLOOR(procTime() TO HOURS)}} will be moved into a {{LogicalProject}} (or
{{LogicalCalc}}) to create a new attribute. A following {{LogicalAggregate}} will then use
that attribute as a grouping column.

Once we detect such a pattern, we have to rewrite the plan and replace the {{LogicalAggregate}}
and parts of the {{LogicalProject}} by a  {{LogicalWindowAggregate}}. The {{LogicalWindowAggregate}}
includes a window definition. Depending on which marker function is used ({{rowtime}} or {{proctime}})
the window definition is either for a processing or an event time window. After the translation,
the function is no longer available.

With this approach, we can only translate very specific queries. However, I don't think we
can easily provide a generic translation for SQL window queries.
With Calcite's special window functions TUMBLE, HOP, and SESSION (see Calcite-1345 and http://calcite.apache.org/docs/stream.html)
the detection of windowing patterns will be much easier.

> Support tumbling window on streaming tables in the SQL API
> ----------------------------------------------------------
>
>                 Key: FLINK-5624
>                 URL: https://issues.apache.org/jira/browse/FLINK-5624
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API & SQL
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>
> This is a follow up of FLINK-4691.
> FLINK-4691 adds supports for group-windows for streaming tables. This jira proposes to
expose the functionality in the SQL layer via the {{GROUP BY}} clauses, as described in http://calcite.apache.org/docs/stream.html#tumbling-windows.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message