hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11388) there should only be 1 Initiator for compactions per Hive installation
Date Mon, 21 Dec 2015 19:44:46 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066964#comment-15066964
] 

Alan Gates commented on HIVE-11388:
-----------------------------------

Ok, that makes sense.  Since the row will never really appear in the database then the hostname
field may not make sense.  A couple of other thoughts:
# This "lock" will be limited by any intervening commits.  This is probably ok as we probably
shouldn't be locking across commits anyway, but we want to make sure we aren't inadvertently
dropping the lock midway through.
# Can we make this a couple of protected methods in TxnHandler {{void getMutex(String mutexName)}}
and {{void releaseMutex(String mutexName)}}.  This way when the HBaseMetastore work gets around
to implementing TxnHandler and has to do it in a completely different way it will still have
internal method to do it with.  If this adds a bunch of work you can ignore the request.

> there should only be 1 Initiator for compactions per Hive installation
> ----------------------------------------------------------------------
>
>                 Key: HIVE-11388
>                 URL: https://issues.apache.org/jira/browse/HIVE-11388
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 1.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Critical
>
> org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs inside the metastore
service to manage compactions of ACID tables.  There should be exactly 1 instance of this
thread (even with multiple Thrift services).
> This is documented in https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration
but not enforced.
> Should add enforcement, since more than 1 Initiator could cause concurrent attempts to
compact the same table/partition - which will not work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message