spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Mateus Pires <dmate...@gmail.com>
Subject Blog post: DataFrame.transform -- Spark function composition
Date Wed, 05 Jun 2019 20:47:15 GMT
Hi everyone!

I just published this blog post on how Spark Scala custom transformations
can be re-arranged to better be composed and used within .transform:


https://medium.com/@dmateusp/dataframe-transform-spark-function-composition-eb8ec296c108

I found the discussions in this group to be largely around issues /
performance, this post only concerns itself with code readability,
hopefully not off-topic!

I welcome any feedback I can get :)

Daniel

Mime
View raw message