spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject [SQL] Understanding RewriteCorrelatedScalarSubquery optimization (and TreeNode.transform)
Date Sun, 27 May 2018 19:43:24 GMT
Hi,

I'm trying to understand RewriteCorrelatedScalarSubquery optimization and
how extractCorrelatedScalarSubqueries [1] works. I don't understand how
"The expression is rewritten and returned." is done. How is the expression
rewritten?

Since it's private it's not even possible to write tests and that got me
thinking how you go about code like this? How do you know whether it works
fine or not? Any help? I'd appreciate.

[1]
https://github.com/apache/spark/blob/branch-2.3/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala?utf8=%E2%9C%93#L290-L299

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski

Mime
View raw message