hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-21731) Hive import fails, post upgrade of source 3.0 cluster, to a target 4.0 cluster with strict managed table set to true.
Date Fri, 17 May 2019 17:55:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-21731?focusedWorklogId=244164&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-244164
]

ASF GitHub Bot logged work on HIVE-21731:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 17/May/19 17:54
            Start Date: 17/May/19 17:54
    Worklog Time Spent: 10m 
      Work Description: sankarh commented on pull request #628: HIVE-21731 : Hive import fails,
post upgrade of source 3.0 cluster, to a target 4.0 cluster with strict managed table set
to true.
URL: https://github.com/apache/hive/pull/628#discussion_r285227479
 
 

 ##########
 File path: itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/WarehouseInstance.java
 ##########
 @@ -563,7 +563,21 @@ public void testEventCounts(String dbName, long fromEventId, Long toEventId,
Int
   }
 
   public boolean isAcidEnabled() {
-    return hiveConf.getBoolVar(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY);
+    if (hiveConf.getBoolVar(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY) &&
+        hiveConf.getVar(HiveConf.ConfVars.HIVE_TXN_MANAGER).equals("org.apache.hadoop.hive.ql.lockmgr.DbTxnManager"))
{
+      return true;
+    }
+    return false;
+  }
+
+  public void disableAcid() {
+    hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY, false);
+    hiveConf.setVar(HiveConf.ConfVars.HIVE_TXN_MANAGER, "org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager");
+  }
+
+  public void enableAcid() {
+    hiveConf.setBoolVar(HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY, true);
+    hiveConf.setVar(HiveConf.ConfVars.HIVE_TXN_MANAGER, "org.apache.hadoop.hive.ql.lockmgr.DbTxnManager");
 
 Review comment:
   sure. Can we add a comment about this in this method? So that later someone need not wonder
why is it not working.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 244164)
    Time Spent: 2h 50m  (was: 2h 40m)

> Hive import fails, post upgrade of source 3.0 cluster, to a target 4.0 cluster with strict
managed table set to true.
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-21731
>                 URL: https://issues.apache.org/jira/browse/HIVE-21731
>             Project: Hive
>          Issue Type: Bug
>            Reporter: mahesh kumar behera
>            Assignee: mahesh kumar behera
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-21731.01.patch, HIVE-21731.02.patch, HIVE-21731.03.patch, HIVE-21731.04.patch
>
>          Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> The scenario is 
>  # Replication policy is set with hive  3.0 source cluster (strict managed table set
to false) and hive 4.0 target cluster with strict managed table set  true.
>  # User upgrades the 3.0 source cluster to 4.0 cluster using upgrade tool.
>  # The upgrade converts all managed tables to acid tables.
>  # In the next repl dump, user sets hive .repl .dump .include .acid .tables and hive
.repl .bootstrap. acid. tables set true triggering bootstrap of newly converted ACID tables.
>  # As the old tables are non-txn tables, dump is not filtering the events even tough
bootstrap acid table is set to true. This is causing the repl load to fail as the write id
is not set in the table object.
>  # If we ignore the event replay, the bootstrap is failing with dump directory mismatch
error.
> The fix should be 
>  # Ignore dumping the alter table event if bootstrap acid table is set true and the alter
is converting a non-acid table to acid table.
>  # In case of bootstrap during incremental load, ignore the dump directory property set
in table object.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message