spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ReeceRobinson <>
Subject Re: Do I really need to build Spark for Hive/Thrift Server support?
Date Tue, 28 Jul 2015 21:46:52 GMT
I am building an analytics environment based on Spark and want to use HIVE in
multi-user mode i.e. not use the embedded derby database but use Postgres
and HDFS instead. I am using the included Spark Thrift Server to process
queries using Spark SQL.

The documentation gives me the impression that I need to create a custom
build of Spark 1.4.1. However I don't think this is either accurate now OR
it is for some different context I'm not aware of?

I used the pre-built Spark 1.4.1 distribution today with my hive-site.xml
for Postgres and HDFS and it worked! I see the warehouse files turn up in
HDFS and I see the metadata inserted into Postgres when I created a test

I can connect to the Thrift Server using beeline and perform queries on my
data. I also verified using the Spark UI that the SQL is being processed by
Spark SQL.

So I guess I'm asking is the document out-of-date or am I missing something?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message