hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pengcheng Xiong (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-16797) Enhance HiveFilterSetOpTransposeRule to remove union branches
Date Mon, 05 Jun 2017 17:43:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037265#comment-16037265
] 

Pengcheng Xiong edited comment on HIVE-16797 at 6/5/17 5:42 PM:
----------------------------------------------------------------

I use pull-up-constant to pull the constant out of union. Then use RexSimplify to simplify
that to see if it can be reduced to always false. However, there are still several comments
regarding patch 02: (1) it sounds like that I was not able to simplify (($2=1 OR $2=2) AND
$2=3) to a false. Here ($2=1 OR $2=2) comes from two branches of union. Thus, I introduce
a hive union merge rule (btw, calcite union merge rule does not fire well in current hive
master) so that we can check  ($2=1 AND $2=3) and ($2=2 AND $2=3), repectively, which works
with RexSimplify. (2) it sounds like RexSimplify also can not reduce ($2>2 AND $2=3) to
false. There is a test case in filter_union.q for that and you will see. (3) if we can assume
that it is always a  project under union, we may have better options. (4) for tpcds queries,
current patch is good enough.


was (Author: pxiong):
I use pull-up-constant to pull the constant out of union. Then use RexSimplify to simplify
that to see if it can be reduced to always false. However, there are still several comments
regarding patch 02: (1) it sounds like that I was not able to simplify (($2=1 OR $2=2) AND
$2=3) to a false. Here ($2=1 OR $2=2) comes from two branches of union. Thus, I introduce
a hive union merge rule (btw, calcite union merge rule does not fire well in current hive
master) so that we can check  ($2=1 AND $2=3) and ($2=2 AND $2=3), repectively, which works
with RexSimplify. (2) it sounds like RexSimplify also can not reduce ($2>2 AND $2=3) to
false. There is a test case in filter_union.q for that and you will see. (3) if we can assume
that it is always a  project under union, we may have better options.

> Enhance HiveFilterSetOpTransposeRule to remove union branches
> -------------------------------------------------------------
>
>                 Key: HIVE-16797
>                 URL: https://issues.apache.org/jira/browse/HIVE-16797
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Pengcheng Xiong
>            Assignee: Pengcheng Xiong
>         Attachments: HIVE-16797.01.patch, HIVE-16797.02.patch
>
>
> in query4.q, we can see that it creates a CTE with union all of 3 branches. Then it is
going to do a 3 way self-join of the CTE with predicates. The predicates actually specifies
only one of the branch in CTE to participate in the join. Thus, in some cases, e.g.,
> {code}
>                /- filter(false) -----TS0 
> union all  - filter(false) -----TS1
>                \-----TS2
> {code}
> we can cut the branches of TS0 and TS1. The union becomes only TS2.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message