spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Mahadevan <ar...@apache.org>
Subject Re: Plans for Session Windows?
Date Thu, 09 Aug 2018 17:12:44 GMT
There is no stock API to do this directly, but it can be implemented on top
of mapGroupWithState like here -


https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredSessionization.scala



It would be worth bundling this into a builtin api, but AFAIK there are no
plans yet.



Thanks,

Arun


On 9 August 2018 at 08:02, Mike Sukmanowsky <mike.sukmanowsky@gmail.com>
wrote:

> Hi all,
>
> Just wondering if Spark has any plans to support session-based windows in
> structured streaming as documented by Apache Beam here
> <https://beam.apache.org/documentation/programming-guide/#provided-windowing-functions>
or
> perhaps better in this blog post
> <https://www.oreilly.com/ideas/the-world-beyond-batch-streaming-102>?
>
> [image: Figure-16---Session-Merging-a526d52186fcc33f3b5b5c59b176ac4e.jpg]
>
> --
> Mike Sukmanowsky
> Aspiring Digital Carpenter
>
> *e*: mike.sukmanowsky@gmail.com
>
> LinkedIn <http://www.linkedin.com/profile/view?id=10897143> | github
> <https://github.com/msukmanowsky>
>
>

Mime
View raw message