spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arnaud Wolf <arnaudeddy.w...@gmail.com>
Subject Enable hql on the JDBC thrift server
Date Fri, 10 Nov 2017 07:08:04 GMT
Hi everyone.

I try to build a middleware that executes injection-safe dynamic SQL 
queries using Spark, with Cassandra as storage engine. Results of 
executed queries need to be stored into a newly created Cassandra table.

Spark SQL does not provide any feature for safe parameter binding, so I 
thought about using the JDBC thrift server and the JDBC interface.

Inserting data into an external table from hive is performed by running 
CREATE EXTERNAL TABLE <table_name>... STORED BY...

However, when trying to execute this statement through the thrift 
server, I always get the following error:

java.sql.SQLException: org.apache.spark.sql.catalyst.parser.ParseException:
Operation not allowed: STORED BY(line 1, pos 31)

After some web digging, I could find that the reason was that the thrift 
server was using Spark SQL and that Hive support is disabled by default 
on Spark SQL. Then the common recommendation is to manually create a 
SparkSession and use enableHiveSupport().

The problem is that I don't want to do that as I can't do safe query 
binding with Spark SQL. I want to use only the JDBC interface. Do you 
know if there is a way to enable Hive support at thrift server startup?

If there is no way, can somebody recommend a safe way for parameter binding?

Thank you for your consideration, and sorry for the long question.


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message