hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
Date Tue, 12 Jun 2018 23:59:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510408#comment-16510408
] 

Sergey Shelukhin commented on HIVE-19838:
-----------------------------------------

Hmm, I thought I could repro TestTxnNoBucketsVectorized failure, but this test fails for me
even without this patch, due to rows in the beginning of testCTAS
{noformat}
    runStatementOnDriver("create table myctas stored as ORC TBLPROPERTIES ('transactional"
+
      "'='true', 'transactional_properties'='default') as select a, b from " + Table.NONACIDORCTBL);
{noformat}
being in reverse order w.r.t. files (rows are the same but the bucket_00000 row is in 00001
and vice versa).
This is not the same failure as above by the looks of it.
Retrying the patch.

cc [~ekoifman] 

> simplify & fix ColumnizedDeleteEventRegistry load loop
> ------------------------------------------------------
>
>                 Key: HIVE-19838
>                 URL: https://issues.apache.org/jira/browse/HIVE-19838
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>            Priority: Major
>         Attachments: HIVE-19838.01.patch, HIVE-19838.02.patch, HIVE-19838.patch
>
>
> Apparently sometimes the delete count in ACID stats doesn't match what merger actually
returns.
> It could be due to some deltas having duplicate deletes from parallel queries (I guess?)
that are being squashed by the merger or some other reasons beyond my mortal comprehension.
> The loop assumes the merger will return the exact number of records, so it fails with
array index exception. Also, it could actually be done in a single loop.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message