spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hyukjin Kwon (JIRA)" <>
Subject [jira] [Commented] (SPARK-26835) Document configuration properties of Spark SQL Generic Load/Save Functions
Date Thu, 07 Feb 2019 07:27:00 GMT


Hyukjin Kwon commented on SPARK-26835:

They are explained in API documentation. "Generic Load/Save Functions section of Spark SQL
documentation " is like a general guide. It doesn't necessarily document everything.
CSV documentation isn't there due to documentation duplication concern. 

Please go ahead and fix it if you're willing to do. Otherwise, I would leave this just closed
to leave only active JIRAs. All the documentations can be improved in all the ways.

> Document configuration properties of Spark SQL Generic Load/Save Functions
> --------------------------------------------------------------------------
>                 Key: SPARK-26835
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: Peter Horvath
>            Priority: Major
> Currently the [Generic Load/Save Functions section of Spark SQL documentation|]

>  does not explain the available configuration properties at all.
> Neither the available formats nor their configuration properties are listed properly:
there are some usage samples, but that is really.
> For some formats, there is a remark to visit the site of the provider; quote: "_To find
more detailed information about the extra ORC/Parquet options, visit the official Apache ORC/Parquet
> However, this is not applicable for all format providers; for example there is not even
a hint regarding the CSV writer's configuration properties or where they can be looked up.
> Please add documentation regarding the configuration properties. Either copy over documentation
completely, or link.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message