spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "" <>
Subject Re: New to Spark
Date Wed, 02 Dec 2015 01:52:09 GMT
Which version spark in your use case ? You made hive metastore to be used by Spark, 
that mean you can run sql queries over the current hive tables , right ? Or you just use 
local hive metastore embeded in spark sql side ? I think you need to provide more info
for your spark sql and hive config, that would help to locate root cause for the problem.

From: Ashok Kumar
Date: 2015-12-01 18:54
Subject: New to Spark


I am new to Spark.

I am trying to use spark-sql with SPARK CREATED and HIVE CREATED tables.

I have successfully made Hive metastore to be used by Spark.

In spark-sql I can see the DDL for Hive tables. However, when I do select count(1) from HIVE_TABLE
it always returns zero rows.

If I create a table in spark as create table SPARK_TABLE as select * from HIVE_TABLE, the
table schema is created but no data.

I can then use Hive to do INSET SELECT from HIVE_TABLE into SPARK_TABLE. That works.

I can then use spark-sql to query the table.

My questions:

Is this correct that spark-sql only sees data in spark created tables but not any data in
Hive tables?
How can I make Spark read data from existing Hive tables.


View raw message