spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jianshi Huang (JIRA)" <>
Subject [jira] [Created] (SPARK-6561) Add partition support in saveAsParquet
Date Fri, 27 Mar 2015 06:21:52 GMT
Jianshi Huang created SPARK-6561:

             Summary: Add partition support in saveAsParquet
                 Key: SPARK-6561
             Project: Spark
          Issue Type: Improvement
          Components: SQL
    Affects Versions: 1.3.0, 1.3.1
            Reporter: Jianshi Huang

Now ParquetRelation2 supports automatic partition discovery which is very nice. 

When we save a DataFrame into Parquet files, we also want to have it partitioned.

The proposed API looks like this:

def saveAsParquet(path: String, partitionColumns: Seq[String])


This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message