Jungtaek Lim created SPARK-29312:
------------------------------------
Summary: Inconsistent behavior on metrics when dropping rows in state between
FlatMapGroupWithState and other stateful operators
Key: SPARK-29312
URL: https://issues.apache.org/jira/browse/SPARK-29312
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.0.0
Reporter: Jungtaek Lim
Even rows are evicted from state, we don't increase the value of "updated" metrics. The rule
applies on streaming aggregations as well as streaming join, but FlatMapGroupsWithState increases
the value of "updated" metrics, allowing the metrics to be shown like (total = 0, updated
= 1) which looks weird for who are familiar with metrics with others.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org
|