spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Baretta <alexbare...@gmail.com>
Subject RE: Unsupported Catalyst types in Parquet
Date Mon, 29 Dec 2014 16:13:53 GMT
Daoyuan,

Thanks for creating the jiras. I need these features by... last week, so
I'd be happy to take care of this myself, if only you or someone more
experienced than me in the SparkSQL codebase could provide some guidance.

Alex
On Dec 29, 2014 12:06 AM, "Wang, Daoyuan" <daoyuan.wang@intel.com> wrote:

> Hi Alex,
>
> I'll create JIRA SPARK-4985 for date type support in parquet, and
> SPARK-4987 for timestamp type support. For decimal type, I think we only
> support decimals that fits in a long.
>
> Thanks,
> Daoyuan
>
> -----Original Message-----
> From: Alessandro Baretta [mailto:alexbaretta@gmail.com]
> Sent: Saturday, December 27, 2014 2:47 PM
> To: dev@spark.apache.org; Michael Armbrust
> Subject: Unsupported Catalyst types in Parquet
>
> Michael,
>
> I'm having trouble storing my SchemaRDDs in Parquet format with SparkSQL,
> due to my RDDs having having DateType and DecimalType fields. What would it
> take to add Parquet support for these Catalyst? Are there any other
> Catalyst types for which there is no Catalyst support?
>
> Alex
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message