spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aesha Dhar Roy <aesha....@gmail.com>
Subject Support for group aggregate pandas UDF in streaming aggregation for SPARK 3.0 python
Date Tue, 11 Aug 2020 14:28:57 GMT
Hi,

Is there any plan to remove the limitation mentioned below?

*Streaming aggregation doesn't support group aggregate pandas UDF *

We want to run our data modelling jobs real time using Spark 3.0 and kafka
2.4 and need to have support for custom aggregate pandas UDF on stream
windows.
Is there any plan for this in the upcoming releases ?

Regards,
Aesha

Mime
View raw message