spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Horvath (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-26835) Document configuration properties of Spark SQL Generic Load/Save Functions
Date Thu, 07 Feb 2019 11:34:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-26835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16762584#comment-16762584
] 

Peter Horvath commented on SPARK-26835:
---------------------------------------

This might be trivial / self-explanatory if you have been working with Spark for a while,
but for newcomers, it's a bit confusing.

Having better / more details docs is always good. :)

 

I've opened a pull request for this: [https://github.com/apache/spark/pull/23742]

 

> Document configuration properties of Spark SQL Generic Load/Save Functions
> --------------------------------------------------------------------------
>
>                 Key: SPARK-26835
>                 URL: https://issues.apache.org/jira/browse/SPARK-26835
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: Peter Horvath
>            Priority: Major
>
> Currently the [Generic Load/Save Functions section of Spark SQL documentation|https://spark.apache.org/docs/2.4.0/sql-data-sources-load-save-functions.html]

>  does not explain the available configuration properties at all.
> Neither the available formats nor their configuration properties are listed properly:
there are some usage samples, but that is really.
> For some formats, there is a remark to visit the site of the provider; quote: "_To find
more detailed information about the extra ORC/Parquet options, visit the official Apache ORC/Parquet
websites."_
> However, this is not applicable for all format providers; for example there is not even
a hint regarding the CSV writer's configuration properties or where they can be looked up.
> Please add documentation regarding the configuration properties. Either copy over documentation
completely, or link.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message