hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-22039) Query with CBO crashes HS2 in corner case
Date Fri, 26 Jul 2019 03:47:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16893296#comment-16893296
] 

Rajesh Balamohan commented on HIVE-22039:
-----------------------------------------

Just wanted to mention that CALCITE-2247 might be just breaking the loop. Need to check why
the loop is there in the first place.

> Query with CBO crashes HS2 in corner case
> -----------------------------------------
>
>                 Key: HIVE-22039
>                 URL: https://issues.apache.org/jira/browse/HIVE-22039
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO
>    Affects Versions: 3.1.1, 2.3.4
>            Reporter: Rajesh Balamohan
>            Priority: Major
>         Attachments: image-2019-07-25-04-52-20-002.png, image-2019-07-25-04-52-33-686.png
>
>
> Here is a very simple repro for this case. Query would crash HS2. Temp workaround is
to turn off CBO for such corner cases.
> It runs into a infinite loop creating too many number of RexCalls and finally OOMs.
> This is observed in 2.x, 3.x.
> With 4.x (master branch), it does not happen. Master has {{calcite-core-1.19.0.jar}},
where as 3.x has {{calcite-core-1.16.0.jar}}.
> I haven't investigated yet on which specific bug in calcite is triggering this (if I
move hive-master to {{calcite-core-1.16.0.jar}}, this problem shows up in 4.x as well)
>  
> {noformat}
> drop table if exists tableA;
> drop table if exists tableB;
> create table if not exists tableA(id int, reporting_date string) stored as orc;
> create table if not exists tableB(id int, reporting_date string) partitioned by (datestr
string) stored as orc;
> explain with tableA_cte as (
>     select
>         id,
>         reporting_date
>     from tableA
>   ),
> tableA_cte_2 as (
>     select
>         0 as id,
>         reporting_date
>     from tableA
>   ),
> tableA_cte_5 as (
>   select * from tableA_cte
>   union 
>   select * from tableA_cte_2  
> ),
> tableB_cte_0 as (
>     select
>         id,
>         reporting_date
>     from tableB   
>     where reporting_date  = '2018-10-29'    
>   ),
> tableB_cte_1 as (
>     select
>         0 as id,
>         reporting_date
>     from tableB  
>     where datestr = '2018-10-29'  
>   ),
> tableB_cte_4 as (
>     select * from tableB_cte_0
>     union 
>     select * from tableB_cte_1    
>   )
> select
>   a.id as id,
>   b.reporting_date
> from tableA_cte_5 a
> join tableB_cte_4 b on (a.id = b.id and a.reporting_date = b.reporting_date);
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message