spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Terry Kim <yumin...@gmail.com>
Subject Re: dataFrame.na.fill() fails for column with dot
Date Tue, 09 Feb 2021 18:33:18 GMT
You probably need to update f. name here
<https://github.com/apache/spark/blob/18b30107adb37d3c7a767a20cc02813f0fdb86da/sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala#L413>
as
well, but we can discuss further when you create a JIRA/PR.

Thanks,
Terry

On Tue, Feb 9, 2021 at 9:53 AM Terry Kim <yuminkim@gmail.com> wrote:

> Thanks Amandeep. This seems like a valid bug to me as quoted columns are
> not handled property for na.fill(). I think the better place to fix is in
> DataFrameNaFunctions.scala
> <https://github.com/apache/spark/blob/18b30107adb37d3c7a767a20cc02813f0fdb86da/sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala#L422>
where
> "f.name" should be quoted.
>
> Could you create a JIRA
> <https://issues.apache.org/jira/projects/SPARK/issues>?
>
> Thanks,
> Terry
>

Mime
View raw message