flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3609) Revisit selection of Calcite rules
Date Wed, 16 Mar 2016 13:10:33 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15197276#comment-15197276
] 

ASF GitHub Bot commented on FLINK-3609:
---------------------------------------

Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1797#discussion_r56329514
  
    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/api/table/plan/rules/FlinkRuleSets.scala
---
    @@ -29,50 +29,45 @@ object FlinkRuleSets {
         */
       val DATASET_OPT_RULES: RuleSet = RuleSets.ofList(
     
    -    // filter rules
    +    // push a filter into a join
         FilterJoinRule.FILTER_ON_JOIN,
    +    // push filter into the children of a join
         FilterJoinRule.JOIN,
    -    FilterMergeRule.INSTANCE,
    -    FilterAggregateTransposeRule.INSTANCE,
    +    // push filter through an aggregation
    +    FlinkFilterAggregateTransposeRule.INSTANCE,
     
    -    // push and merge projection rules
    +    // aggregation and projection rules
         AggregateProjectMergeRule.INSTANCE,
    -    ProjectMergeRule.INSTANCE,
    +    AggregateProjectPullUpConstantsRule.INSTANCE,
    +    // push a projection past a filter or vice versa
         ProjectFilterTransposeRule.INSTANCE,
         FilterProjectTransposeRule.INSTANCE,
    -    AggregateProjectPullUpConstantsRule.INSTANCE,
    -    JoinPushExpressionsRule.INSTANCE,
    +    // push a projection to the children of a join
         ProjectJoinTransposeRule.INSTANCE,
    +    // remove identity project
         ProjectRemoveRule.INSTANCE,
    +    // reorder sort and projection
         SortProjectTransposeRule.INSTANCE,
         ProjectSortTransposeRule.INSTANCE,
     
    -    // merge and push unions rules
    -    // TODO: Add a rule to enforce binary unions
    +    // join rules
    +    JoinPushExpressionsRule.INSTANCE,
    +
    +    // remove union with only a single child
         UnionEliminatorRule.INSTANCE,
    -    FlinkJoinUnionTransposeRule.LEFT_UNION,
    -    FlinkJoinUnionTransposeRule.RIGHT_UNION,
    --- End diff --
    
    These rules would push a join into a union. This means that a join is executed on each
input of the union and the join results are unioned afterwards. Additional joins are likely
to cause higher resource consumption. For example if the unioned inputs are fed into the probe-side
of a join, the build side would need to be replicated. 
    
    Unless we have very good statistics and a more fine-grained cost model, I would not try
to optimize and trust the user (at least for the Table API).


> Revisit selection of Calcite rules
> ----------------------------------
>
>                 Key: FLINK-3609
>                 URL: https://issues.apache.org/jira/browse/FLINK-3609
>             Project: Flink
>          Issue Type: Task
>          Components: Table API
>            Reporter: Fabian Hueske
>            Assignee: Fabian Hueske
>
> We should revisit the selection of Calcite rules, e.g., remove rule to reorder joins
or join inputs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message