hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Vary (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14138) CBO failed for select current_database()
Date Thu, 30 Jun 2016 15:48:10 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15357309#comment-15357309
] 

Peter Vary commented on HIVE-14138:
-----------------------------------

[~jcamachorodriguez] Could you please take a look at this? Every time one of the above queries
is run, the error is logged, and the execution falls back to the normal mode without CBO.
I was able to come up a patch which turns of the CBO for every query where there is no source
table, but not sure that this is the right solution, or even the problem :)

{noformat}
===================================================================
--- ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java	(revision de3d86cdd3db174d6bc5d8c65796a1e981171124)
+++ ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java	(revision )
@@ -387,8 +387,9 @@
     boolean needToLogMessage = STATIC_LOG.isInfoEnabled();
     boolean isSupportedRoot = root == HiveParser.TOK_QUERY || root == HiveParser.TOK_EXPLAIN
         || qb.isCTAS();
-    boolean isSupportedType = qb.getIsQuery() || qb.isCTAS()
-        || cboCtx.type == PreCboCtx.Type.INSERT;
+    // Queries without a source table currently are not supported by CBO
+    boolean isSupportedType = (qb.getIsQuery() && !qb.containsQueryWithoutSourceTable())
+        || qb.isCTAS() || cboCtx.type == PreCboCtx.Type.INSERT;
     boolean noBadTokens = HiveCalciteUtil.validateASTForUnsupportedTokens(ast);
     boolean result = isSupportedRoot && isSupportedType && getCreateViewDesc()
== null
         && noBadTokens;
@@ -400,7 +401,7 @@
           msg += "doesn't have QUERY or EXPLAIN as root and not a CTAS; ";
         }
         if (!isSupportedType) {
-          msg += "is not a query, CTAS, or insert; ";
+          msg += "is not a query with at least one source table os subquery, CTAS, or insert;
";
         }
         if (getCreateViewDesc() != null) {
           msg += "has create view; ";
===================================================================
--- ql/src/java/org/apache/hadoop/hive/ql/parse/QBExpr.java	(revision de3d86cdd3db174d6bc5d8c65796a1e981171124)
+++ ql/src/java/org/apache/hadoop/hive/ql/parse/QBExpr.java	(revision )
@@ -120,4 +120,17 @@
     }
     return qbexpr1.isSimpleSelectQuery() && qbexpr2.isSimpleSelectQuery();
   }
+
+  /**
+   * returns true, if the query block contains any query, or subquery without a source table
+   * Like select current_user(), select current_database()
+   * @return true, if the query block contains any query without a source table
+   */
+  public boolean containsQueryWithoutSourceTable() {
+    if (qb != null) {
+      return qb.containsQueryWithoutSourceTable();
+    } else {
+      return qbexpr1.containsQueryWithoutSourceTable() || qbexpr2.containsQueryWithoutSourceTable();
+    }
+  }
 }
===================================================================
--- ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java	(revision de3d86cdd3db174d6bc5d8c65796a1e981171124)
+++ ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java	(revision )
@@ -437,4 +437,17 @@
     return aliasInsideView;
   }
 
+  /**
+   * returns true, if the query block contains any query, or subquery without a source table
+   * Like select current_user(), select current_database()
+   * @return true, if the query block contains any query without a source table
+   */
+  public boolean containsQueryWithoutSourceTable() {
+    for (QBExpr qbexpr : aliasToSubq.values()) {
+      if (qbexpr.containsQueryWithoutSourceTable()) {
+        return true;
+      }
+    }
+    return aliasToTabs.size()==0 && aliasToSubq.size()==0;
+  }
 }
{noformat}

> CBO failed for select current_database()
> ----------------------------------------
>
>                 Key: HIVE-14138
>                 URL: https://issues.apache.org/jira/browse/HIVE-14138
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO
>            Reporter: Peter Vary
>            Priority: Minor
>
> When issuing the following query, with hive.cbo.enable set to true:
> select current_database();
> The following exception is printed to the Hiveserver2 logs:
> 2016-06-30T09:58:24,146 ERROR [HiveServer2-Handler-Pool: Thread-33] parse.CalcitePlanner:
CBO failed, skipping CBO. 
> org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException: Unsupported
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:3136)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:940)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:894)
> 	at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113)
> 	at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:969)
> 	at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
> 	at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:712)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:280)
> 	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10795)
> 	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:239)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:250)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:438)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:329)
> 	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1159)
> 	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1146)
> 	at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:191)
> 	at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276)
> 	at org.apache.hive.service.cli.operation.Operation.run(Operation.java:324)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:464)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:451)
> 	at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:295)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:509)
> 	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
> 	at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
> 	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> 	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> 	at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
> 	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message