spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (Jira)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-25555) Generic constructs for windowing and support for custom windows
Date Mon, 16 Mar 2020 22:52:07 GMT

     [ https://issues.apache.org/jira/browse/SPARK-25555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dongjoon Hyun updated SPARK-25555:
----------------------------------
    Affects Version/s:     (was: 3.0.0)
                       3.1.0

> Generic constructs for windowing and support for custom windows
> ---------------------------------------------------------------
>
>                 Key: SPARK-25555
>                 URL: https://issues.apache.org/jira/browse/SPARK-25555
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 3.1.0
>            Reporter: Arun Mahadevan
>            Priority: Major
>
> Refactor windows logic with generic "assignWindows" and "mergeWindows" constructs. The
existing windows (Tumbling and sliding) and generic session windows can be built on top of
this. It could be extended to support different types of custom windowing. 
> K,Values -> AssignWindows (produces [k, v, timestamp, window]) -> GroupByKey (shuffle)
-> MergeWindows (optional step) -> GroupWindows -> aggregate values.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message