flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6516) using real row count instead of dummy row count when optimizing plan
Date Fri, 14 Jul 2017 09:26:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087085#comment-16087085

ASF GitHub Bot commented on FLINK-6516:

Github user fhueske commented on a diff in the pull request:

    --- Diff: flink-libraries/flink-table/src/main/scala/org/apache/flink/table/plan/nodes/PhysicalTableSourceScan.scala
    @@ -70,4 +73,18 @@ abstract class PhysicalTableSourceScan(
       def copy(traitSet: RelTraitSet, tableSource: TableSource[_]): PhysicalTableSourceScan
    +  override def estimateRowCount(mq: RelMetadataQuery): Double = {
    +    val tableSourceTable = getTable.unwrap(classOf[TableSourceTable[_]])
    +    if (tableSourceTable.getStatistic != FlinkStatistic.UNKNOWN) {
    --- End diff --
    Right, but in the current implementation the statistics of the `TableSourceTable` are
preferred over the stats of the `TableSource` (`if (tableSourceTable.getStatistic != FlinkStatistic.UNKNOWN)
{ return rowcount; }`).
    If I see this correctly, the stats of the `TableSourceTable` are the stats in the catalog,
but a `FilterableTableSource` knows if a filter has been pushed down and should have more
accurate stats because it can apply the selectivity of the pushed down filter. 
    So in my opinion the `TableSource` statistics should be preferred over the static `TableSourceTable`
    Is my reasoning correct or did I overlook something @godfreyhe?
    Thanks, Fabian

> using real row count instead of dummy row count when optimizing plan
> --------------------------------------------------------------------
>                 Key: FLINK-6516
>                 URL: https://issues.apache.org/jira/browse/FLINK-6516
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table API & SQL
>            Reporter: godfrey he
>            Assignee: godfrey he
> Currently, the statistic of {{TableSourceTable}} is {{UNKNOWN}} mostly, and the statistic
from {{ExternalCatalog}} maybe is null also. Actually, only each {{TableSource}} knows its
statistic exactly, especial for {{FilterableTableSource}} and {{PartitionableTableSource}}.
So we can add {{getTableStats}} method in {{TableSource}}, and use it in TableSourceScan's
estimateRowCount method to get real row count.

This message was sent by Atlassian JIRA

View raw message