hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-24072) HiveAggregateJoinTransposeRule may try to create an invalid transformation
Date Thu, 27 Aug 2020 15:32:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-24072?focusedWorklogId=475355&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-475355
]

ASF GitHub Bot logged work on HIVE-24072:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 27/Aug/20 15:31
            Start Date: 27/Aug/20 15:31
    Worklog Time Spent: 10m 
      Work Description: kgyrtkirk commented on a change in pull request #1432:
URL: https://github.com/apache/hive/pull/1432#discussion_r478508344



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveAggregateJoinTransposeRule.java
##########
@@ -145,8 +145,7 @@ public void onMatch(RelOptRuleCall call) {
         int fieldCount = joinInput.getRowType().getFieldCount();
         final ImmutableBitSet fieldSet =
             ImmutableBitSet.range(offset, offset + fieldCount);
-        final ImmutableBitSet belowAggregateKeyNotShifted =
-            belowAggregateColumns.intersect(fieldSet);

Review comment:
       @jcamachor 
   I think this intersect is not needed; it cause some trouble in case
   * grouped by say 1,3,4
   * columns 1,2,3 is coming from left side of join; column 2 is the join key
   * the intersect will remove column 2 ; which will induce a shift in the mapping - and will
rotate in the join key into a projkect later on
   
   I think in calcite the following is true:
   for a join betwen inputs: 
   * T1(a,b,c)
   * T2(d,e,f)
   the output is always a,b,c,d,e,f - independently the join condition
   so I see no rational need for this `intersect`...it was here for a few years now...and
the class "borned" with this line in it...




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 475355)
    Time Spent: 20m  (was: 10m)

> HiveAggregateJoinTransposeRule may try to create an invalid transformation
> --------------------------------------------------------------------------
>
>                 Key: HIVE-24072
>                 URL: https://issues.apache.org/jira/browse/HIVE-24072
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zoltan Haindrich
>            Assignee: Zoltan Haindrich
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> {code}
> java.lang.AssertionError: 
> Cannot add expression of different type to set:
> set type is RecordType(INTEGER NOT NULL o_orderkey, DECIMAL(10, 0) o_totalprice, DATE
o_orderdate, INTEGER NOT NULL c_custkey, VARCHAR(25) CHARACTER SET "UTF-16LE" c_name, DOUBLE
$f5) NOT NULL
> expression type is RecordType(INTEGER NOT NULL o_orderkey, INTEGER NOT NULL o_custkey,
DECIMAL(10, 0) o_totalprice, DATE o_orderdate, INTEGER NOT NULL c_custkey, DOUBLE $f1) NOT
NULL
> set is rel#567:HiveAggregate.HIVE.[].any(input=HepRelVertex#490,group={2, 4, 5, 6, 7},agg#0=sum($1))
> expression is HiveProject(o_orderkey=[$2], o_custkey=[$3], o_totalprice=[$4], o_orderdate=[$5],
c_custkey=[$6], $f1=[$1])
>   HiveJoin(condition=[=($2, $0)], joinType=[inner], algorithm=[none], cost=[{2284.5 rows,
0.0 cpu, 0.0 io}])
>     HiveAggregate(group=[{0}], agg#0=[sum($1)])
>       HiveProject(l_orderkey=[$0], l_quantity=[$4])
>         HiveTableScan(table=[[tpch_0_001, lineitem]], table:alias=[l])
>     HiveJoin(condition=[=($0, $6)], joinType=[inner], algorithm=[none], cost=[{1.0000000000009115E15
rows, 0.0 cpu, 0.0 io}])
>       HiveJoin(condition=[=($4, $1)], joinType=[inner], algorithm=[none], cost=[{1650.0
rows, 0.0 cpu, 0.0 io}])
>         HiveProject(o_orderkey=[$0], o_custkey=[$1], o_totalprice=[$3], o_orderdate=[$4])
>           HiveTableScan(table=[[tpch_0_001, orders]], table:alias=[orders])
>         HiveProject(c_custkey=[$0], c_name=[$1])
>           HiveTableScan(table=[[tpch_0_001, customer]], table:alias=[customer])
>       HiveProject($f0=[$0])
>         HiveFilter(condition=[>($1, 3E2)])
>           HiveAggregate(group=[{0}], agg#0=[sum($4)])
>             HiveTableScan(table=[[tpch_0_001, lineitem]], table:alias=[lineitem])
> 	at org.apache.calcite.plan.RelOptUtil.verifyTypeEquivalence(RelOptUtil.java:383)
> 	at org.apache.calcite.plan.hep.HepRuleCall.transformTo(HepRuleCall.java:57)
> 	at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:236)
> 	at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveAggregateJoinTransposeRule.onMatch(HiveAggregateJoinTransposeRule.java:300)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message