spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Armbrust (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4943) Parsing error for query with table name having dot
Date Wed, 31 Dec 2014 20:26:13 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14262412#comment-14262412
] 

Michael Armbrust commented on SPARK-4943:
-----------------------------------------

One possible approach here is to change the signature of UnresolvedRelation as follows:

{code}
case class UnresolvedRelation(tableIdentifier: Seq[String], alias: Option[String])
{code}

This way we can leave parsing and handling of backticks up to the parser and let the catalogs
interpret the identifiers in a system dependent way.

> Parsing error for query with table name having dot
> --------------------------------------------------
>
>                 Key: SPARK-4943
>                 URL: https://issues.apache.org/jira/browse/SPARK-4943
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.2.0
>            Reporter: Alex Liu
>
> When integrating Spark 1.2.0 with Cassandra SQL, the following query is broken. It was
working for Spark 1.1.0 version. Basically we use the table name having dot to include database
name 
> {code}
> [info]   java.lang.RuntimeException: [1.29] failure: ``UNION'' expected but `.' found
> [info] 
> [info] SELECT test1.a FROM sql_test.test1 AS test1 UNION DISTINCT SELECT test2.a FROM
sql_test.test2 AS test2
> [info]                             ^
> [info]   at scala.sys.package$.error(package.scala:27)
> [info]   at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(SparkSQLParser.scala:33)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:79)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$1.apply(SQLContext.scala:79)
> [info]   at org.apache.spark.sql.catalyst.SparkSQLParser$$anonfun$org$apache$spark$sql$catalyst$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:174)
> [info]   at org.apache.spark.sql.catalyst.SparkSQLParser$$anonfun$org$apache$spark$sql$catalyst$SparkSQLParser$$others$1.apply(SparkSQLParser.scala:173)
> [info]   at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136)
> [info]   at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1$$anonfun$apply$2.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$Failure.append(Parsers.scala:202)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$Parser$$anonfun$append$1.apply(Parsers.scala:254)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
> [info]   at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
> [info]   at scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890)
> [info]   at scala.util.parsing.combinator.PackratParsers$$anon$1.apply(PackratParsers.scala:110)
> [info]   at org.apache.spark.sql.catalyst.AbstractSparkSQLParser.apply(SparkSQLParser.scala:31)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$parseSql$1.apply(SQLContext.scala:83)
> [info]   at org.apache.spark.sql.SQLContext$$anonfun$parseSql$1.apply(SQLContext.scala:83)
> [info]   at scala.Option.getOrElse(Option.scala:120)
> [info]   at org.apache.spark.sql.SQLContext.parseSql(SQLContext.scala:83)
> [info]   at org.apache.spark.sql.cassandra.CassandraSQLContext.cassandraSql(CassandraSQLContext.scala:53)
> [info]   at org.apache.spark.sql.cassandra.CassandraSQLContext.sql(CassandraSQLContext.scala:56)
> [info]   at com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply$mcV$sp(CassandraSQLSpec.scala:169)
> [info]   at com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply(CassandraSQLSpec.scala:168)
> [info]   at com.datastax.spark.connector.sql.CassandraSQLSpec$$anonfun$20.apply(CassandraSQLSpec.scala:168)
> [info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
> [info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
> [info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> [info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
> [info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
> [info]   at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)
> [info]   at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
> [info]   at org.scalatest.FlatSpec.withFixture(FlatSpec.scala:1683)
> [info]   at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
> [info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> [info]   at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)
> [info]   at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
> [info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
> [info]   at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
> [info]   at scala.collection.immutable.List.foreach(List.scala:318)
> [info]   at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
> [info]   at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
> [info]   at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
> [info]   at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)
> [info]   at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)
> [info]   at org.scalatest.Suite$class.run(Suite.scala:1424)
> [info]   at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
> [info]   at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
> [info]   at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
> [info]   at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)
> [info]   at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)
> [info]   at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:466)
> [info]   at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:677)
> [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:294)
> [info]   at sbt.ForkMain$Run$2.call(ForkMain.java:284)
> [info]   at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> [info]   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [info]   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [info]   at java.lang.Thread.run(Thread.java:745)
> [info] - should allow to select rows with union distinct clause *** FAILED *** (46 milliseconds)
> [info]   java.lang.RuntimeException: [1.29] failure: ``UNION'' expected but `.' found
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message