hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-21880) Enable flaky test TestReplicationScenariosAcidTablesBootstrap.testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites.
Date Mon, 01 Jul 2019 08:02:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-21880?focusedWorklogId=270023&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-270023
]

ASF GitHub Bot logged work on HIVE-21880:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/Jul/19 08:01
            Start Date: 01/Jul/19 08:01
    Worklog Time Spent: 10m 
      Work Description: maheshk114 commented on pull request #684: HIVE-21880 : Enable flaky
test TestReplicationScenariosAcidTablesBootstrap.testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites.
URL: https://github.com/apache/hive/pull/684#discussion_r298922242
 
 

 ##########
 File path: standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
 ##########
 @@ -3236,18 +3238,28 @@ public NotificationEventResponse getNextNotification(long lastEventId,
int maxEv
     NotificationEventResponse filtered = new NotificationEventResponse();
     if (rsp != null && rsp.getEvents() != null) {
       long nextEventId = lastEventId + 1;
+      long prevEventId = lastEventId;
       for (NotificationEvent e : rsp.getEvents()) {
+        LOG.debug("Got event with id : " + e.getEventId());
         if (e.getEventId() != nextEventId) {
-          LOG.error("Requested events are found missing in NOTIFICATION_LOG table. Expected:
{}, Actual: {}. "
-                  + "Probably, cleaner would've cleaned it up. "
-                  + "Try setting higher value for hive.metastore.event.db.listener.timetolive.
"
-                  + "Also, bootstrap the system again to get back the consistent replicated
state.",
-                  nextEventId, e.getEventId());
-          throw new IllegalStateException(REPL_EVENTS_MISSING_IN_METASTORE);
+          if (e.getEventId() == prevEventId) {
+            LOG.error("NOTIFICATION_LOG table has multiple events with the same event Id
{}. " +
+                    "Something went wrong when inserting notification events.  Bootstrap
the system " +
+                    "again to get back teh consistent replicated state.", prevEventId);
 
 Review comment:
   i am not sure if we need to handle this separately. As per code, this can never happen
now.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 270023)
    Time Spent: 1h 20m  (was: 1h 10m)

> Enable flaky test TestReplicationScenariosAcidTablesBootstrap.testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites.
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-21880
>                 URL: https://issues.apache.org/jira/browse/HIVE-21880
>             Project: Hive
>          Issue Type: Bug
>          Components: repl
>    Affects Versions: 4.0.0
>            Reporter: Sankar Hariappan
>            Assignee: Ashutosh Bapat
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-21880.01.patch, HIVE-21880.02.patch, HIVE-21880.03.patch
>
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Need tp enable TestReplicationScenariosAcidTablesBootstrap.testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites
which is disabled as it is flaky and randomly failing with below error.
> {code}
> Error Message
> Notification events are missing in the meta store.
> Stacktrace
> java.lang.IllegalStateException: Notification events are missing in the meta store.
> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getNextNotification(HiveMetaStoreClient.java:3246)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
> 	at com.sun.proxy.$Proxy58.getNextNotification(Unknown Source)
> 	at org.apache.hadoop.hive.ql.metadata.events.EventUtils$MSClientNotificationFetcher.getNextNotificationEvents(EventUtils.java:107)
> 	at org.apache.hadoop.hive.ql.metadata.events.EventUtils$NotificationEventIterator.fetchNextBatch(EventUtils.java:159)
> 	at org.apache.hadoop.hive.ql.metadata.events.EventUtils$NotificationEventIterator.hasNext(EventUtils.java:189)
> 	at org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.incrementalDump(ReplDumpTask.java:231)
> 	at org.apache.hadoop.hive.ql.exec.repl.ReplDumpTask.execute(ReplDumpTask.java:121)
> 	at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> 	at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
> 	at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2709)
> 	at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2361)
> 	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2028)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1788)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1782)
> 	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:162)
> 	at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:223)
> 	at org.apache.hadoop.hive.ql.parse.WarehouseInstance.run(WarehouseInstance.java:227)
> 	at org.apache.hadoop.hive.ql.parse.WarehouseInstance.dump(WarehouseInstance.java:282)
> 	at org.apache.hadoop.hive.ql.parse.WarehouseInstance.dump(WarehouseInstance.java:265)
> 	at org.apache.hadoop.hive.ql.parse.WarehouseInstance.dump(WarehouseInstance.java:289)
> 	at org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTablesBootstrap.testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites(TestReplicationScenariosAcidTablesBootstrap.java:328)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> 	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> 	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> 	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> {code}
> https://builds.apache.org/job/PreCommit-HIVE-Build/17591/testReport/org.apache.hadoop.hive.ql.parse/TestReplicationScenariosAcidTablesBootstrap/testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message