spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandeep Katta (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-32764) compare of -0.0 < 0.0 return true
Date Thu, 03 Sep 2020 17:48:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190331#comment-17190331
] 

Sandeep Katta edited comment on SPARK-32764 at 9/3/20, 5:47 PM:
----------------------------------------------------------------

[~cloud_fan] and [~smilegator]

I did some analysis on this, it is because as a part of SPARK-30009 org.apache.spark.util.Utils.nanSafeCompareDoubles
is replaced with java.lang.Double.compare. Same can be confirmed from the codegen code, I
have attached those for referrence.

 

If you see the implementation of java.lang.Double.compare
 
*java.lang.Double.compare(-0.0, 0.0) < 0 evaluates to true*

*java.lang.Double.compare(0.0, -0.0) < 0 evaluates to false*

 


was (Author: sandeep.katta2007):
[~cloud_fan] and [~smilegator]

I did some analysis on this, it is because as a part of SPARK-30009 org.apache.spark.util.Utils.nanSafeCompareDoubles
is replaced with java.lang.Double.compare. Same can be confirmed from the codegen code, I
have attached those for referrence.

 

If you see the implementation of java.lang.Double.compare
 
*java.lang.Double.compare(-0.0, 0.0) < 0 evaluates to true*

*java.lang.Double.compare(0.0, -0.0) < 0 evaluates to false*

 

> compare of -0.0 < 0.0 return true
> ---------------------------------
>
>                 Key: SPARK-32764
>                 URL: https://issues.apache.org/jira/browse/SPARK-32764
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.0, 3.0.1
>            Reporter: Izek Greenfield
>            Priority: Major
>              Labels: correctness
>         Attachments: 2.4_codegen.txt, 3.0_Codegen.txt
>
>
> {code:scala}
>  val spark: SparkSession = SparkSession
>       .builder()
>       .master("local")
>       .appName("SparkByExamples.com")
>       .getOrCreate()
>     spark.sparkContext.setLogLevel("ERROR")
>     import spark.sqlContext.implicits._
>     val df = Seq((-0.0, 0.0)).toDF("neg", "pos")
>       .withColumn("comp", col("neg") < col("pos"))
>       df.show(false)
> ======
> +----+---+----+
> |neg |pos|comp|
> +----+---+----+
> |-0.0|0.0|true|
> +----+---+----+{code}
> I think that result should be false.
> **Apache Spark 2.4.6 RESULT**
> {code}
> scala> spark.version
> res0: String = 2.4.6
> scala> Seq((-0.0, 0.0)).toDF("neg", "pos").withColumn("comp", col("neg") < col("pos")).show
> +----+---+-----+
> | neg|pos| comp|
> +----+---+-----+
> |-0.0|0.0|false|
> +----+---+-----+
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message