hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11948) Investigate TxnHandler and CompactionTxnHandler to see where we improve concurrency
Date Mon, 16 Nov 2015 22:32:10 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007498#comment-15007498
] 

Alan Gates commented on HIVE-11948:
-----------------------------------

In TxnHandler, around line 495:
bq. Why doesn't this get a txnid as parameter?  The caller should either know the txnid or
know there isn't one.  Either way getTxnIdFromLockId() will not be needed.  This would be
a Thrift change.
We should file a JIRA for that.  Same goes for comment at line 501.  We might just want to
file an umbrella JIRA saying "take care of TODOs in TxnHandler and CompactionTxnHandler" and
then we can file JIRAs for individual ones.

TxnHandler, line 522:
{code}
     if (txnid > 0) {
        heartbeatTxn(dbConn, txnid);
     }
    else {
        heartbeatLock(dbConn, extLockId);
    }
{code}

Previously the code was:
{code}
 heartbeatLock(dbConn, extLockId);		
 ...
 if (txnid > 0)  heartbeatTxn(dbConn, txnid);
{code}
You've changed the logic so that locks will only be heartbeat if there is no transaction.
 I don't think that's what you want.

TxnHander unlock(), around line 581, you moved the check that a lock is associated with a
txn below the failure detection.  Are you depending on the db constraints to catch that the
lock entry can't be deleted because a txn it is associated with still exists?  If so, that
should be commented.  If not, this is a logical error as we want to make sure never to unlock
a lock associated with a txn.

TxnHandler.getRequiredIsolationLevel(), line 2270
{code}
if(dbProduct == null) {
  Connection tmp = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
  determineDatabaseProduct(tmp);
  closeDbConn(tmp);
}
{code}
We should modify determineDatabaseProduct to accept null for the connection and create its
own rather than repeating this logic anytime we don't have a connection.



> Investigate TxnHandler and CompactionTxnHandler to see where we improve concurrency
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-11948
>                 URL: https://issues.apache.org/jira/browse/HIVE-11948
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 0.14.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>         Attachments: HIVE-11948.3.patch, HIVE-11948.4.patch, HIVE-11948.5.patch, HIVE-11948.6.patch,
HIVE-11948.7.patch, HIVE-11948.patch
>
>
> at least some operations (or parts of operations) can run at READ_COMMITTED.
> CompactionTxnHandler.setRunAs()
> CompactionTxnHandler.findNextToCompact()
> if update stmt includes cq_state = '" + INITIATED_STATE + "'" in WHERE clause and logic
to look for "next" candidate
> CompactionTxnHandler.markCompacted()
> perhaps add cq_state=WORKING_STATE in Where clause (mostly as an extra consistency check)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message