spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yin Huai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-6315) SparkSQL 1.3.0 (RC3) fails to read parquet file generated by 1.1.1
Date Fri, 13 Mar 2015 14:17:38 GMT

    [ https://issues.apache.org/jira/browse/SPARK-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14360399#comment-14360399
] 

Yin Huai commented on SPARK-6315:
---------------------------------

Should we change the target version to 1.3.1?

> SparkSQL 1.3.0 (RC3) fails to read parquet file generated by 1.1.1
> ------------------------------------------------------------------
>
>                 Key: SPARK-6315
>                 URL: https://issues.apache.org/jira/browse/SPARK-6315
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.3.0
>            Reporter: Michael Armbrust
>            Assignee: Cheng Lian
>            Priority: Blocker
>
> Parquet files generated by Spark 1.1 have a deprecated representation of the schema.
 In Spark 1.3 we fail to read these files through the new Parquet code path.  We should continue
to read these files until we formally deprecate this representation.
> As a workaround:
> {code}
> SET spark.sql.parquet.useDataSourceApi=false
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message