spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jianshi Huang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-6561) Add partition support in saveAsParquet
Date Fri, 27 Mar 2015 06:21:52 GMT
Jianshi Huang created SPARK-6561:
------------------------------------

             Summary: Add partition support in saveAsParquet
                 Key: SPARK-6561
                 URL: https://issues.apache.org/jira/browse/SPARK-6561
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 1.3.0, 1.3.1
            Reporter: Jianshi Huang


Now ParquetRelation2 supports automatic partition discovery which is very nice. 

When we save a DataFrame into Parquet files, we also want to have it partitioned.

The proposed API looks like this:

{code}
def saveAsParquet(path: String, partitionColumns: Seq[String])
{code}

Jianshi



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message