spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yash Datta (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-6471) Metastoreschema should only be a subset of parquetSchema to support dropping of columns using replace columns
Date Mon, 23 Mar 2015 17:49:10 GMT
Yash Datta created SPARK-6471:
---------------------------------

             Summary: Metastoreschema should only be a subset of parquetSchema to support
dropping of columns using replace columns
                 Key: SPARK-6471
                 URL: https://issues.apache.org/jira/browse/SPARK-6471
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 1.3.0
            Reporter: Yash Datta
             Fix For: 1.4.0


Currently in the parquet relation 2 implementation, error is thrown in case merged schema
is not exactly the same as metastore schema. 
But to support cases like deletion of column using replace column command, we can relax the
restriction so that even if metastore schema is a subset of merged parquet schema, the query
will work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message