flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8492) [FLINK-8492][table] Fix calc cost bug
Date Thu, 25 Jan 2018 15:58:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16339417#comment-16339417
] 

ASF GitHub Bot commented on FLINK-8492:
---------------------------------------

Github user fhueske commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5347#discussion_r163878119
  
    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/nodes/CommonCalc.scala
---
    @@ -149,26 +152,42 @@ trait CommonCalc {
         // conditions, etc. We only want to account for computations, not for simple projections.
         // CASTs in RexProgram are reduced as far as possible by ReduceExpressionsRule
         // in normalization stage. So we should ignore CASTs here in optimization stage.
    -    val compCnt = calcProgram.getExprList.asScala.toList.count {
    -      case _: RexInputRef => false
    -      case _: RexLiteral => false
    -      case c: RexCall if c.getOperator.getName.equals("CAST") => false
    -      case _ => true
    -    }
    +    // Also, we add 1 to take calc RelNode number into consideration, so the cost of
merged calc
    +    // RelNode will less than the total cost of un-merged calcs.
    +    val compCnt = calcProgram.getExprList.asScala.toList.count(isCom(_)) + 1
     
    -    val newRowCnt = estimateRowCount(calcProgram, rowCnt)
    +    val newRowCnt = estimateRowCount(rexBuilder, calcProgram, rowCnt)
         planner.getCostFactory.makeCost(newRowCnt, newRowCnt * compCnt, 0)
       }
     
       private[flink] def estimateRowCount(
    +      rexBuilder: RexBuilder,
    --- End diff --
    
    we don't need the `RexBuilder` if we get the condition `RexNode` from the `RexProgram`.


> [FLINK-8492][table] Fix calc cost bug
> -------------------------------------
>
>                 Key: FLINK-8492
>                 URL: https://issues.apache.org/jira/browse/FLINK-8492
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API &amp; SQL
>            Reporter: Hequn Cheng
>            Assignee: Hequn Cheng
>            Priority: Major
>
> Considering the following test, unsupported exception will be thrown due to multi
calc existing between correlate and TableFunctionScan.
> {code:java}
> // code placeholder
> @Test
> def testCrossJoinWithMultiFilter(): Unit = {
>   val t = testData(env).toTable(tEnv).as('a, 'b, 'c)
>   val func0 = new TableFunc0
>   val result = t
>     .join(func0('c) as('d, 'e))
>     .select('c, 'd, 'e)
>     .where('e > 10)
>     .where('e > 20)
>     .select('c, 'd)
>     .toAppendStream[Row]
>   result.addSink(new StreamITCase.StringSink[Row])
>   env.execute()
>   val expected = mutable.MutableList("Jack#22,Jack,22", "Anna#44,Anna,44")
>   assertEquals(expected.sorted, StreamITCase.testResults.sorted)
> }
> {code}
> I can see two options to fix this problem:
>  # Adapt calcite OptRule to merge the continuous calc.
>  # Merge multi calc in correlate convert rule.
> I prefer the second one, not only it is easy to implement but also i think with or without
an optimize rule should not influence flink functionality. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message