[ https://issues.apache.org/jira/browse/SPARK-26829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16766939#comment-16766939
]
Marco Gaido commented on SPARK-26829:
-------------------------------------
You can set the output column name and you can rename it as you want after the transformation.
The only question which I think this ticket poses is: why are we currently forbidding to override
an existing column in several places (eg. PCA, MinMaxScaler, StandardScaler)? This seems to
be a consistent behavior and there are easy workarounds, so I am not sure if it's worth changing
it.
> In place standard scaler so the column remains same after transformation
> ------------------------------------------------------------------------
>
> Key: SPARK-26829
> URL: https://issues.apache.org/jira/browse/SPARK-26829
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Affects Versions: 2.3.2
> Reporter: Santokh Singh
> Priority: Major
>
> Standard scaler and some similar transformations takes input column name and produce
a new column, either accepting output column or generating new one with some random name after
performing transformation.
> "inplace" flag on true does not generate new column in output in dataframe after
transformation; preserves schema of df.
> "inplace" flag on false works the way its currently working.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org
|