hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-23039) Checkpointing for repl dump bootstrap phase
Date Wed, 01 Apr 2020 10:24:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23039?focusedWorklogId=413957&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-413957
]

ASF GitHub Bot logged work on HIVE-23039:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/Apr/20 10:23
            Start Date: 01/Apr/20 10:23
    Worklog Time Spent: 10m 
      Work Description: anishek commented on pull request #965: HIVE-23039 Checkpointing for
repl dump bootstrap phase
URL: https://github.com/apache/hive/pull/965#discussion_r401486847
 
 

 ##########
 File path: itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosExternalTables.java
 ##########
 @@ -906,6 +908,131 @@ public void replicationWithTableNameContainsKeywords() throws Throwable
{
             .verifyReplTargetProperty(replicatedDbName);
   }
 
+  @Test
+  public void testCheckPointing() throws Throwable {
+    List<String> withClauseOptions = externalTableBasePathWithClause();
+    WarehouseInstance.Tuple bootstrapDump = primary.run("use " + primaryDbName)
+            .run("CREATE TABLE t1(a string) STORED AS TEXTFILE")
+            .run("CREATE EXTERNAL TABLE t2(a string) STORED AS TEXTFILE")
+            .run("insert into t1 values (1)")
+            .run("insert into t1 values (2)")
+            .run("insert into t2 values (11)")
+            .run("insert into t2 values (21)")
+            .dump(primaryDbName, withClauseOptions);
+
+    // verify that the external table info is written correctly for bootstrap
+    assertExternalFileInfo(Arrays.asList("t2"), bootstrapDump.dumpLocation, primaryDbName);
 
 Review comment:
   might be better to use another location here completely, sometimes since we are writing
to same dump location and also the db directory is there you might have files in locations
that are slight off and you wont realize it till you run actual prod scenario
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 413957)
    Time Spent: 20m  (was: 10m)

> Checkpointing for repl dump bootstrap phase
> -------------------------------------------
>
>                 Key: HIVE-23039
>                 URL: https://issues.apache.org/jira/browse/HIVE-23039
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Aasha Medhi
>            Assignee: Aasha Medhi
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-23039.01.patch
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message