spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Armbrust (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-3851) Support for reading parquet files with different but compatible schema
Date Mon, 02 Feb 2015 20:20:34 GMT

     [ https://issues.apache.org/jira/browse/SPARK-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Michael Armbrust updated SPARK-3851:
------------------------------------
    Assignee: Cheng Lian

> Support for reading parquet files with different but compatible schema
> ----------------------------------------------------------------------
>
>                 Key: SPARK-3851
>                 URL: https://issues.apache.org/jira/browse/SPARK-3851
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Michael Armbrust
>            Assignee: Cheng Lian
>            Priority: Critical
>
> Right now it is required that all of the parquet files have the same schema.  It would
be nice to support some safe subset of cases where the schemas of files is different.  For
example:
>  - Adding and removing nullable columns.
>  - Widening types (a column that is of both Int and Long type)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message