spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacek Laskowski (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-20122) Analyzer reports "unresolved operator 'Aggregate" for nonexistent window specifications
Date Tue, 28 Mar 2017 06:35:41 GMT
Jacek Laskowski created SPARK-20122:
---------------------------------------

             Summary: Analyzer reports "unresolved operator 'Aggregate" for nonexistent window
specifications
                 Key: SPARK-20122
                 URL: https://issues.apache.org/jira/browse/SPARK-20122
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 2.2.0
            Reporter: Jacek Laskowski
            Priority: Minor


{{Analyzer}} should be more elaborate on the root cause of the {{org.apache.spark.sql.AnalysisException}}
when {{WindowSpecReference}} cannot be resolved.

I think the real issue is that {{AstBuilder}} creates a lookup table for {{WindowSpecDefinition}}s
per name and when it cannot resolve {{WindowSpecReference}} it simply fails analysis (it's
too early in the SQL parsing).

{code}
scala> table.selectExpr("count() OVER nonexistentWindowSpec")
org.apache.spark.sql.AnalysisException: unresolved operator 'Aggregate [unresolvedwindowexpression(count(),
WindowSpecReference(nonexistentWindowSpec)) AS unresolvedwindowexpression(count())#15];;
'Aggregate [unresolvedwindowexpression(count(), WindowSpecReference(nonexistentWindowSpec))
AS unresolvedwindowexpression(count())#15]
+- LocalRelation <empty>, [value#11]

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:40)
  at org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:90)
  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$3.apply(CheckAnalysis.scala:408)
  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$3.apply(CheckAnalysis.scala:407)
  at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:127)
  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:407)
  at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:90)
  at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:53)
  at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:67)
  at org.apache.spark.sql.Dataset.org$apache$spark$sql$Dataset$$withPlan(Dataset.scala:2832)
  at org.apache.spark.sql.Dataset.select(Dataset.scala:1137)
  at org.apache.spark.sql.Dataset.selectExpr(Dataset.scala:1172)
  ... 48 elided
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message