hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15988) LLAP: cache ORC ACID delete deltas
Date Wed, 08 Mar 2017 14:51:37 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15901370#comment-15901370
] 

Gopal V commented on HIVE-15988:
--------------------------------

[~teddy.choi]: the delete deltas pose a deadlock possibility - since they are not opened one
by one, but all at the same time, they have the possibility of locking out the entire cache
under some circumstances (i.e all deltas are open, but not all have been consumed).

This is not the case with the insert deltas, since only one will be opened per split at a
time, with the previous one being closed before the next one is opened - the eviction behaviour
is much more easy to model.

> LLAP: cache ORC ACID delete deltas
> ----------------------------------
>
>                 Key: HIVE-15988
>                 URL: https://issues.apache.org/jira/browse/HIVE-15988
>             Project: Hive
>          Issue Type: Improvement
>          Components: llap, Transactions
>            Reporter: Teddy Choi
>            Assignee: Teddy Choi
>         Attachments: HIVE-15988.1.patch
>
>
> HIVE-12631 implemented ORC ACID table support in LLAP. If we cache ACID delete delta
in LLAP, it will be faster.
> The cache is currently low-level (CB-level in ORC), so we could just use it to read bases
and deltas (deltas should be cached with higher priority) and merge as usual. We could also
cache merged representation in future.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message