spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivier Girardot <ssab...@gmail.com>
Subject Re: coalesce on dataFrame
Date Wed, 01 Jul 2015 10:05:26 GMT
PySpark or Spark (scala) ?
When you use coalesce with anything but a column you must use a literal
like that in PySpark :

from pyspark.sql import functions as F

F.coalesce(df.a, F.lit(True))

Le mer. 1 juil. 2015 à 12:03, Ewan Leith <ewan.leith@realitymine.com> a
écrit :

> It's in spark 1.4.0, or should be at least:
>
> https://issues.apache.org/jira/browse/SPARK-6972
>
> Ewan
>
> -----Original Message-----
> From: Hafiz Mujadid [mailto:hafizmujadid00@gmail.com]
> Sent: 01 July 2015 08:23
> To: user@spark.apache.org
> Subject: coalesce on dataFrame
>
> How can we use coalesce(1, true) on dataFrame?
>
>
> Thanks
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/coalesce-on-dataFrame-tp23564.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional
> commands, e-mail: user-help@spark.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message