spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ferran Galí <>
Subject Troubles interacting with different version of Hive metastore
Date Fri, 25 Sep 2015 11:02:08 GMT

I'm trying to start the SparkSQL thriftserver over YARN, connecting it to
the 1.1.0-cdh5.4.3 hive metastore that we already have in production.

I downloaded the latest version of Spark (1.5.0), I just followed the
instructions from the documentation
and I configured the spark-default.conf file with the following parameters:

*spark.sql.hive.metastore.version* 1.1.0

Moreover, I've also placed the *hive-site.xml* into the *conf* folder.
When I startup the thriftserver, it eventually crashes with the following

15/09/25 12:50:57 INFO hive.HiveContext: default warehouse location is
15/09/25 12:50:57 INFO hive.HiveContext: Initializing
HiveMetastoreConnection version 1.1.0 using [;@51fd1e7f
15/09/25 12:50:58 INFO client.ClientWrapper: Inspected Hadoop version:
Exception in thread "main" java.lang.ClassNotFoundException:
java.lang.NoClassDefFoundError: com/google/common/base/Predicate when
creating Hive client using classpath: *HERE ALL THE JARS INSIDE THE
(including guava-14.0.1.jar)*

If I put "maven" inside the *spark.sql.hive.metastore.jars* parameter I get
the same error.

Do you know what could I be doing wrong? I don't understand, because the
class that he's complaining ( it's already
in the classpath.

Ferran Galí i Reniu
[image: Trovit] <>
[image: Twitter] <>[image: Facebook]
<>[image: Linkedin]
<>[image: Google +]
<>[image: Blog]
*Ferran Galí i Reniu*
+34 93 209 2556
Avda. Diagonal 601, 9ª
08028, Barcelona

View raw message