hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Yeom (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
Date Sat, 21 Oct 2017 00:25:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16213569#comment-16213569
] 

Steve Yeom commented on HIVE-17856:
-----------------------------------

Talked with Sergey. 
Sergey indicated that the current IOW execution code/path for MM tables is not ACID compliant

because first it deletes directories as mentioned above by an IOW writer/updater.

For one case, to support Snapshot isolation by concurrent open reader transactions, we should
not delete them with 
the writers when possibly open transactions may exist, but through a mechanism which is sync
with open transactions, 
as in the case of ACID tables, where delta directories are merged/deleted by compactors with
TxnList 
with open and aborted xacts. 

> MM tables - IOW is not ACID compliant
> -------------------------------------
>
>                 Key: HIVE-17856
>                 URL: https://issues.apache.org/jira/browse/HIVE-17856
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Transactions
>            Reporter: Sergey Shelukhin
>            Assignee: Steve Yeom
>              Labels: mm-gap-1
>
> The following tests were removed from mm_all during "integration"... I should have never
allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas, here we
are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true", "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm; 
> create table iow1_mm(key int) partitioned by (key2 int)  tblproperties("transactional"="true",
"transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1, key from
intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as k1, key +
2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties ("transactional"="true",
"transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message