hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-16669) Fine tune Compaction to take advantage of Acid 2.0
Date Sat, 09 Sep 2017 00:48:01 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135751#comment-16135751
] 

Eugene Koifman edited comment on HIVE-16669 at 9/9/17 12:47 AM:
----------------------------------------------------------------

see todo: TestAcidOnTez.testCtasTezUnion - todo tagged with this HIVE-16669

also note: in the same test, minor compaction creates delete_delta_0000018_0000021  even though
the smallest delete_delta has txn 20.  Tighten the bounds to make file selection more efficient.


was (Author: ekoifman):
see todo: TestAcidOnTez.testCtasTezUnion - todo tagged with this HIVE-16669

> Fine tune Compaction to take advantage of Acid 2.0
> --------------------------------------------------
>
>                 Key: HIVE-16669
>                 URL: https://issues.apache.org/jira/browse/HIVE-16669
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>
> * There is little point using 2.0 vectorized reader since there is no operator pipeline
in compaction
> * If minor compaction just concats delete_delta files together, then the 2 stage compaction
should always ensure that we have a limited number of Orc readers to do the merging and current
OrcRawRecordMerger should be fine
> * ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message