hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Krisztian Kasa (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-24840) Materialized View incremental rebuild produces wrong result set after compaction
Date Wed, 28 Jul 2021 10:23:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-24840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17388669#comment-17388669
] 

Krisztian Kasa commented on HIVE-24840:
---------------------------------------

As far as I know compaction shouldn't affect merge statements since those are translated to
multi insert statements and doesn't depend on the content of COMPLETED_TXN_COMPONENTS table.

> Materialized View incremental rebuild produces wrong result set after compaction
> --------------------------------------------------------------------------------
>
>                 Key: HIVE-24840
>                 URL: https://issues.apache.org/jira/browse/HIVE-24840
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Krisztian Kasa
>            Assignee: Krisztian Kasa
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>          Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> {code}
> create table t1(a int, b varchar(128), c float) stored as orc TBLPROPERTIES ('transactional'='true');
> insert into t1(a,b, c) values (1, 'one', 1.1), (2, 'two', 2.2), (NULL, NULL, NULL);
> create materialized view mat1 stored as orc TBLPROPERTIES ('transactional'='true') as

>             select a,b,c from t1 where a > 0 or a is null;
> delete from t1 where a = 1;
> alter table t1 compact 'major';
> -- Wait until compaction finished.
> alter materialized view mat1 rebuild;
> {code}
> Expected result of query
> {code}
> select * from mat1;
> {code}
> {code}
> 2 two 2
> NULL NULL NULL
> {code}
> but if incremental rebuild is enabled the result is
> {code}
> 1 one 1
> 2 two 2
> NULL NULL NULL
> {code}
> Cause: Incremental rebuild queries whether the source tables of a materialized view has
delete or update transaction since the last rebuild from metastore from COMPLETED_TXN_COMPONENTS
table. However when a major compaction is performed on the source tables the records related
to these tables are deleted from COMPLETED_TXN_COMPONENTS.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message