spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kimahriman <>
Subject Re: Spark hive build and connectivity
Date Thu, 22 Oct 2020 17:48:14 GMT
I have always been a little confused about the different hive-version
integration as well. To expand on this question, we have a Hive 3.1.1
metastore that we can successfully interact with using the -Phive profile
with Hive 2.3.7. We do not use the Hive 3.1.1 jars anywhere in our Spark
applications. Are we just lucky that the 2.3.7 jars are compatible for our
use cases with the 3.1.1 metastore? Or are the
`spark.sql.hive.metastore.jars` only used if you are using a direct JDBC
connection and acting as the metastore?

Also FWIW, the documentation only claims compatibility up to Hive version
3.1.2. Not sure if there's any breaking changes in 3.2 and beyond.

Sent from:

To unsubscribe e-mail:

View raw message