spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guy Khazma <Guy.Kha...@ibm.com>
Subject Re: [DISCUSS] Expensive deterministic UDFs
Date Tue, 31 Dec 2019 13:35:20 GMT
Hi All,

For the use case where the expensive UDF has constant inputs (literals) we
have proposed the following JIRA and PR which calculates the UDF only once
in the driver:
https://issues.apache.org/jira/browse/SPARK-27692
https://github.com/apache/spark/pull/24593

If considering revisiting the optimization for UDFs the above PR can be
considered as well.

Thanks,
Guy



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message