[ https://issues.apache.org/jira/browse/SPARK-26806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-26806:
------------------------------------
Assignee: Shixiong Zhu (was: Apache Spark)
> EventTimeStats.merge doesn't handle "zero.merge(zero)" correctly
> ----------------------------------------------------------------
>
> Key: SPARK-26806
> URL: https://issues.apache.org/jira/browse/SPARK-26806
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.2.1, 2.3.0, 2.3.1, 2.3.2, 2.4.0
> Reporter: Shixiong Zhu
> Assignee: Shixiong Zhu
> Priority: Major
>
> Right now, EventTimeStats.merge doesn't handle "zero.merge(zero)". This will make "avg"
become "NaN". And whatever gets merged with the result of "zero.merge(zero)", "avg" will still
be "NaN". Then finally, "NaN".toLong will return "0" and the user will see the following incorrect
report:
> {code}
> "eventTime" : {
> "avg" : "1970-01-01T00:00:00.000Z",
> "max" : "2019-01-31T12:57:00.000Z",
> "min" : "2019-01-30T18:44:04.000Z",
> "watermark" : "1970-01-01T00:00:00.000Z"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org
|