spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From EarthsonLu <>
Subject [SparkSQL][UDAF] CatalystTypeConverters for each update?
Date Tue, 19 Jul 2016 08:36:28 GMT
I just find that MutableAggregationBuffer.update will convert data for every
update, which is terrible when I use something like Map, Array.

It is hard to implement a collect_set udaf, which will be O(n^2) in this
convert semantic.

Any advice?

View this message in context:
Sent from the Apache Spark Developers List mailing list archive at

To unsubscribe e-mail:

View raw message