spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mihaly Toth (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-23465) Dataset.withAllColumnsRenamed should map all column names to a new one
Date Thu, 05 Apr 2018 15:08:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-23465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mihaly Toth resolved SPARK-23465.
---------------------------------
    Resolution: Won't Fix

Based on PR feedback I would conclude that this functionality is not very much needed.

> Dataset.withAllColumnsRenamed should map all column names to a new one
> ----------------------------------------------------------------------
>
>                 Key: SPARK-23465
>                 URL: https://issues.apache.org/jira/browse/SPARK-23465
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.3.1
>            Reporter: Mihaly Toth
>            Priority: Minor
>
> Currently one can only rename a column only one by one using {{withColumnRenamed()}}
function. When one would like to rename all or most of the columns it would be easier to specify
an algorithm for mapping from the old to the new name (like prefixing) than iterating over
all the fields.
> Example usage is joining to a Dataset with the same or similar schema (special case is
self joining) where the names are the same or overlapping. Such a joined Dataset would fail
at {{saveAsTable()}}
> With the new function usage would be easy like that:
> {code:java}
> ds.withAllColumnsRenamed("prefix" + _)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message