hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-18948) Acquire locks before generating the valid transaction list
Date Wed, 13 Jun 2018 19:27:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511572#comment-16511572
] 

Sergey Shelukhin commented on HIVE-18948:
-----------------------------------------

Discussed a little bit with [~jcamachorodriguez] 
Since we only need this for some optimizers, it's also possible to split optimizers into two
groups and acquire locks and snapshot in the middle.
The 2nd group will not be able to add lockable entities, though (which is anyway the case
if locks are taken before optimizers).

> Acquire locks before generating the valid transaction list
> ----------------------------------------------------------
>
>                 Key: HIVE-18948
>                 URL: https://issues.apache.org/jira/browse/HIVE-18948
>             Project: Hive
>          Issue Type: Improvement
>          Components: Transactions
>    Affects Versions: 3.0.0
>            Reporter: Jesus Camacho Rodriguez
>            Priority: Critical
>
> HIVE-18825 moves the valid transaction list generation logic before query optimization.
In order to support lock-based concurrency control correctly, the logic to acquire the locks
has to be moved before query optimization too, and before the valid transaction list is generated.
> This requires a bit of work/refactoring, since lock acquisition logic relies on read/write
entities, and it is heavily dependent on the QueryPlan object too, e.g., it sets some properties
in the file sink descriptors for the plan. Currently, all these data structures (except for
read entities) are only available after query has been optimized, hence we will need to 1)
generate some of this data structures before query optimization, e.g., write entities, and
2) create and propagate some of the properties so they are set during query optimization,
e.g., those properties contained in the file sink descriptors.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message