spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jianshi Huang <jianshi.hu...@gmail.com>
Subject Loading tables using parquetFile vs. loading tables from Hive metastore with Parquet serde
Date Sun, 15 Feb 2015 17:19:59 GMT
Hi,

If I have a table in Hive metastore saved as Parquet, and I want to use it
in Spark. It seems Spark will use Hive's Parquet serde to load the actual
data.

So is there any difference here? Will predicate pushdown, pruning and
future Parquet optimizations in SparkSQL work for using Hive serde?

Loading tables using parquetFile vs. loading tables from Hive metastore
with Parquet serde


Thanks,
-- 
Jianshi Huang

LinkedIn: jianshi
Twitter: @jshuang
Github & Blog: http://huangjs.github.com/

Mime
View raw message