hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HIVE-23946) Improve control flow and error handling in QTest dataset loading/unloading
Date Mon, 03 Aug 2020 15:57:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-23946?focusedWorklogId=465775&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-465775
]

ASF GitHub Bot logged work on HIVE-23946:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 03/Aug/20 15:56
            Start Date: 03/Aug/20 15:56
    Worklog Time Spent: 10m 
      Work Description: zabetak commented on a change in pull request #1331:
URL: https://github.com/apache/hive/pull/1331#discussion_r464505336



##########
File path: itests/util/src/main/java/org/apache/hadoop/hive/ql/dataset/QTestDatasetHandler.java
##########
@@ -84,23 +83,25 @@ public boolean initDataset(String table, CliDriver cliDriver) throws Exception
{
 
     try {
       CommandProcessorResponse result = cliDriver.processLine(commands);
-      LOG.info("Result from cliDrriver.processLine in initFromDatasets=" + result);
+      LOG.info("Result from cliDrriver.processLine in initDataset=" + result);
     } catch (CommandProcessorException e) {
-      Assert.fail("Failed during initFromDatasets processLine with code=" + e);
+      throw new RuntimeException("Failed while loading table " + table, e);
     }
-
-    return true;
+    // Add the talbe in sources if it is loaded sucessfully
+    addSrcTable(table);
   }
 
-  public boolean unloadDataset(String table, CliDriver cliDriver) throws Exception {
+  private void unloadDataset(String table, CliDriver cliDriver) {
     try {
+      // Remove table from sources otherwise the following command will fail due to EnforceReadOnlyTables.
+      removeSrcTable(table);
       CommandProcessorResponse result = cliDriver.processLine("drop table " + table);
-      LOG.info("Result from cliDrriver.processLine in initFromDatasets=" + result);
+      LOG.info("Result from cliDrriver.processLine in unloadDataset=" + result);
     } catch (CommandProcessorException e) {
-      Assert.fail("Failed during initFromDatasets processLine with code=" + e);
+      // If the unloading fails for any reason then add again the table to sources since
it is still there.
+      addSrcTable(table);
+      throw new RuntimeException("Failed while unloading table " + table, e);

Review comment:
       Given that there are no test failures I would say no but let's wait to see what @kgyrtkirk
has to say :) On the other hand, as I noted in the JIRA there is code that does not work well
if an assertion is thrown here. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 465775)
    Time Spent: 50m  (was: 40m)

> Improve control flow and error handling in QTest dataset loading/unloading
> --------------------------------------------------------------------------
>
>                 Key: HIVE-23946
>                 URL: https://issues.apache.org/jira/browse/HIVE-23946
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Stamatis Zampetakis
>            Assignee: Stamatis Zampetakis
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> This issue focuses mainly on the following methods:
> [QTestDatasetHandler#initDataset| https://github.com/apache/hive/blob/6fbd54c0af60276d49b237defb550938c9c32610/itests/util/src/main/java/org/apache/hadoop/hive/ql/dataset/QTestDatasetHandler.java#L76]
> [QTestDatasetHandler#unloadDataset|https://github.com/apache/hive/blob/6fbd54c0af60276d49b237defb550938c9c32610/itests/util/src/main/java/org/apache/hadoop/hive/ql/dataset/QTestDatasetHandler.java#L95]
> related to QTest dataset loading and unloading.
> The boolean return type in these methods is redundant since they either fail or return
true (they never return false).
> The methods should throw an Exception instead of an AssertionError to indicate failure.
This allows code higher up the stack to perform proper recovery and properly report the failure.
At the moment, if an AssertionError is raised from these methods dependent code (eg., [CoreCliDriver|https://github.com/apache/hive/blob/6fbd54c0af60276d49b237defb550938c9c32610/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreCliDriver.java#L188])
fails to notice that the query has failed. 
> In case of failure in loading/unloading the environment (instance and class variables)
is not properly cleaned leading to failures in all subsequent tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message