hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15988) LLAP: cache ORC ACID delete deltas
Date Wed, 08 Mar 2017 15:06:38 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15901394#comment-15901394
] 

Gopal V commented on HIVE-15988:
--------------------------------

The delete deltas are still really small in memory, so this is more of a hypothetical concern
for a large cache (~4 bits per delete row?).

Looks like we already have a configuration parameter which changes the way ACID deletes are
read into memory, which can also apply to the caching operations - {{hive.transactional.events.mem}}

> LLAP: cache ORC ACID delete deltas
> ----------------------------------
>
>                 Key: HIVE-15988
>                 URL: https://issues.apache.org/jira/browse/HIVE-15988
>             Project: Hive
>          Issue Type: Improvement
>          Components: llap, Transactions
>            Reporter: Teddy Choi
>            Assignee: Teddy Choi
>         Attachments: HIVE-15988.1.patch
>
>
> HIVE-12631 implemented ORC ACID table support in LLAP. If we cache ACID delete delta
in LLAP, it will be faster.
> The cache is currently low-level (CB-level in ORC), so we could just use it to read bases
and deltas (deltas should be cached with higher priority) and merge as usual. We could also
cache merged representation in future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message