spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arnaud Wolf <>
Subject Re: Enable hql on the JDBC thrift server
Date Fri, 10 Nov 2017 07:46:40 GMT
I forgot to precise that I'm using Spark 2.1.0.

On 2017/11/10 16:08, Arnaud Wolf wrote:
> Hi everyone.
> I try to build a middleware that executes injection-safe dynamic SQL 
> queries using Spark, with Cassandra as storage engine. Results of 
> executed queries need to be stored into a newly created Cassandra table.
> Spark SQL does not provide any feature for safe parameter binding, so 
> I thought about using the JDBC thrift server and the JDBC interface.
> Inserting data into an external table from hive is performed by 
> running CREATE EXTERNAL TABLE <table_name>... STORED BY...
> However, when trying to execute this statement through the thrift 
> server, I always get the following error:
> java.sql.SQLException: 
> org.apache.spark.sql.catalyst.parser.ParseException:
> Operation not allowed: STORED BY(line 1, pos 31)
> After some web digging, I could find that the reason was that the 
> thrift server was using Spark SQL and that Hive support is disabled by 
> default on Spark SQL. Then the common recommendation is to manually 
> create a SparkSession and use enableHiveSupport().
> The problem is that I don't want to do that as I can't do safe query 
> binding with Spark SQL. I want to use only the JDBC interface. Do you 
> know if there is a way to enable Hive support at thrift server startup?
> If there is no way, can somebody recommend a safe way for parameter 
> binding?
> Thank you for your consideration, and sorry for the long question.

To unsubscribe e-mail:

View raw message