spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Deprecating UserDefinedGenerator logical operator?
Date Fri, 08 Dec 2017 13:20:41 GMT
Hi,

I've just ran across UserDefinedGenerator logical operator [1] that is used
exclusively for Dataset.explode operator [2][3] that is...

@deprecated("use flatMap() or select() with functions.explode() instead",
"2.0.0")

Could that also "trigger" deprecating UserDefinedGenerator?

[1]
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala#L90

[2]
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2051
[3]
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala#L2092

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski

Mime
View raw message