hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <>
Subject [jira] [Updated] (HIVE-17691) Miscellaneous List
Date Fri, 06 Oct 2017 16:51:00 GMT


Eugene Koifman updated HIVE-17691:
    Summary: Miscellaneous List  (was: Misc Nits)

> Miscellaneous List
> ------------------
>                 Key: HIVE-17691
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Transactions
>            Reporter: Eugene Koifman
> # DDLSemanticAnalyzer.alterTableOutput is unused
> # DDLTask.generateAddMmTasks(Table) - stmtId should probably come from TransactionManager
> # DDLTask.createTable(Hive db, CreateTableDesc crtTbl) has _Long mmWriteId = crtTbl.getInitialMmWriteId();_
logic is unclear..  this ID is only set in one place..
> # FileSinkOperator has multiple places that look like _conf.getWriteType() == AcidUtils.Operation.NOT_ACID
|| conf.isMmTable()_ - what is the writeType for MM tables?  Seems that Wei opted for "work.getLoadTableWork().getWriteType()
!= AcidUtils.Operation.NOT_ACID && !tbd.isMmTable()" to mean MM, e.g. MoveTask.handleStaticParts()
call to Hive.loadPartition()
> # HiveConf.HIVE_TXN_OPERATIONAL_PROPERTIES - the doc/explanation there is obsolete
> # Compactor Initiator likely doesn't work for MM tables.  It's triggered by into in TXN_COMPONENTS/COMPLETED_TXN_COMPONENTS.
 MM tables don't write to either because DbTxnManager.acquireLocks() does  _compBuilder.setIsAcid(AcidUtils.isFullAcidTable(t));_
i.e. it treats MM as non-acid tables
> # In general integration with full Acid seems confused wrt to MM and seems to treat MM
as special table type rather than subtype of Acid table.  (mostly, but not always).
> # LoadSemanticAnalyzer.analyzeInternal(ASTNode) sets statementId to 0 rather than from
> # ImportCommitTask - doesn't currently do anything.  It used to commit mmID.  Need to
verify we properly commit the txn in the Driver
> # As far as I can tell all the mm_*.q tests run on TestCliDriver which means MR.  This
doesn't exercise some code specifically for dealing with writes from Union All queries (CTAS,
Insert into).  On MR this requires "hive.optimize.union.remove=true" (false by default)

This message was sent by Atlassian JIRA

View raw message