spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Sabater Dinter <david.sabater.maill...@gmail.com>
Subject Issues when saving dataframe in Spark 1.4 with parquet format
Date Wed, 01 Jul 2015 10:57:05 GMT
Hi chaps,
It seems there is an issue while saving dataframes in Spark 1.4.

The default file extension inside Hive warehouse folder is now
part-r-X.gz.parquet but while running queries from SparkSQL Thriftserver is
still looking for part-r-X.parquet.

Is there any config parameter we can use as workaround? Is there any Jira
opened about the same? Am I missing anything if I upgraded from Spark 1.3
to 1.4?

The only similar reference I have seen is this:
http://mail-archives.apache.org/mod_mbox/spark-user/201506.mbox/%3CCAHP0wa+JApfVj+Pc2mzWOmzB+MmdOZfbR-XaXDBKOppe68tFww@mail.gmail.com%3E



Thanks.

Mime
View raw message