flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kurt Young (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-11715) Add optimize program to organize optimization phases
Date Fri, 01 Mar 2019 04:17:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-11715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781267#comment-16781267

Kurt Young commented on FLINK-11715:

implemented in 2328ff3a45b889f5bf2c4e8873944980cd904721

> Add optimize program to organize optimization phases
> ----------------------------------------------------
>                 Key: FLINK-11715
>                 URL: https://issues.apache.org/jira/browse/FLINK-11715
>             Project: Flink
>          Issue Type: New Feature
>          Components: SQL / Planner
>            Reporter: godfrey he
>            Assignee: godfrey he
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
> Currently, Flink organizes the optimization phases by different methods in Batch(Stream)TableEnvironment#optimize.
However this is not easy to extend especially there are more than ten optimization stages
in Blink. On the other hand, the methods are very similar, except the match order and rule
sets for hep optimization phases, target traits and rule sets for volcano optimization phases.
> Abstracts each optimization stage into a {{FlinkOptimizeProgram}} in Blink, defined as
> {code}
> /**
>   * Likes [[org.apache.calcite.tools.Program]], FlinkOptimizeProgram transforms a relational
>   * expression into another relational expression.
>   */
> trait FlinkOptimizeProgram[OC <: OptimizeContext] {
>   def optimize(input: RelNode, context: OC): RelNode
> }
> {code}
> {{FlinkOptimizeProgram}}'s subclasses include 
>  1. {{FlinkRuleSetProgram}}, an abstract program can add/remove {{RuleSet}}, set target
>  2. {{FlinkHepRuleSetProgram}}, a subclass of {{FlinkRuleSetProgram}} which runs with
>  3. {{FlinkVolcanoProgram}}, a subclass of {{FlinkRuleSetProgram}} which runs with {{VolcanoPlanner}}.
>  4. {{FlinkGroupProgram}}, a program contains a sequence of sub-programs as a group,
programs in the group will be executed in sequence, and the group can be executed `iterations`
> ......
> {{FlinkChainedPrograms}} is responsible for organizing all the programs, each program's
optimize method will be called in sequence when {{FlinkChainedPrograms}}#optimize is called.

This message was sent by Atlassian JIRA

View raw message