spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pei-Lun Lee <pl...@appier.com>
Subject SparkSQL overwrite parquet file does not generate _common_metadata
Date Thu, 26 Mar 2015 04:48:08 GMT
Hi,

When I save parquet file with SaveMode.Overwrite, it never generate
_common_metadata. Whether it overwrites an existing dir or not.
Is this expected behavior?
And what is the benefit of _common_metadata? Will reading performs better
when it is present?

Thanks,
--
Pei-Lun

Mime
View raw message