spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Liu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot
Date Wed, 31 Dec 2014 21:27:13 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262442#comment-14262442
] 

Alex Liu commented on SPARK-4943:
---------------------------------

The seq can be like 

{code}
tableName, databaseName, clusterName, catalog
{code}

First element is tableName, the next one is databaseName, then clusterName and catalog(option).


SQL query looks like

{code}
SELECT test_table.column1 FROM catalog.cluster.database.table AS test_table 
SELECT test_table.column1 FROM cluster.database.table AS test_table 
SELECT test_table.column1 FROM database.table AS test_table 
SELECT table.column1 FROM table
{code}

Potentially we should not use AS clause here.

Parser only allow maximum four levels in full table name. [catalog].[cluster].[database].[table]

> Parsing error for query with table name having dot
> --------------------------------------------------
>
>                 Key: SPARK-4943
>                 URL: https://issues.apache.org/jira/browse/SPARK-4943
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.2.0
>            Reporter: Alex Liu
>
> When integrating Spark 1.2.0 with Cassandra SQL, the following query is broken. It was
working for Spark 1.1.0 version. Basically we use the table name having dot to include database
name 
> {code}
> [info]   java.lang.RuntimeException: [1.29] failure: ``UNION'' expected but `.' found
> [info] 
> [info] SELECT test1.a FROM sql_test.test1 AS test1 UNION DISTINCT SELECT test2.a FROM
sql_test.test2 AS test2
> [info]                             ^
> [info]   at scala.sys.package$.error(package.scala:27)
> [info]   at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(SparkSQLParser.scala:33)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:79)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:79)
> [info]   at org.apache.spark.sql.catalyst.SparkSQLParser$$anonfun$org$apache$spark$sql$catalyst$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:174)
> [info]   at org.apache.spark.sql.catalyst.SparkSQLParser$$anonfun$org$apache$spark$sql$catalyst$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:173)
> [info]   at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136)
> [info]   at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
> [info]   at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890)
> [info]   at scala.util.parsing.combinator.PackratParsers$$anon$1.apply(PackratParsers.scala:110)
> [info]   at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(SparkSQLParser.scala:31)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$parseSql$1.apply(SQLContext.scala:83)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$parseSql$1.apply(SQLContext.scala:83)
> [info]   at scala.Option.getOrElse(Option.scala:120)
> [info]   at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:83)
> [info]   at org.apache.spark.sql.cassandra.CassandraSQLContext.cassandraSql(CassandraSQLContext.scala:53)
> [info]   at org.apache.spark.sql.cassandra.CassandraSQLContext.sql(CassandraSQLContext.scala:56)
> [info]   at com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply$mcV$sp(CassandraSQLSpec.scala:169)
> [info]   at com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply(CassandraSQLSpec.scala:168)
> [info]   at com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply(CassandraSQLSpec.scala:168)
> [info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
> [info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
> [info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> [info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
> [info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
> [info]   at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)
> [info]   at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
> [info]   at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683)
> [info]   at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
> [info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> [info]   at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)
> [info]   at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
> [info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
> [info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
> [info]   at scala.collection.immutable.List.foreach(List.scala:318)
> [info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
> [info]   at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
> [info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
> [info]   at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)
> [info]   at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)
> [info]   at org.scalatest.Suite$class.run(Suite.scala:1424)
> [info]   at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
> [info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
> [info]   at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)
> [info]   at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)
> [info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:466)
> [info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:677)
> [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:294)
> [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:284)
> [info]   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [info]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [info]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [info]   at java.lang.Thread.run(Thread.java:745)
> [info] - should allow to select rows with union distinct clause *** FAILED *** (46 milliseconds)
> [info]   java.lang.RuntimeException: [1.29] failure: ``UNION'' expected but `.' found
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message