Hi,

After another day trying to get my head around WholeStageCodegenExec and InputAdapter and CollapseCodegenStages optimization rule I came to conclusion that it may have something to do with UnsafeRow vs GenericInternalRow/InternalRow so when a physical operator wants to _somehow_ participate in whole-stage codegen it can extend CodegenSupport trait and enable accessing GenericInternalRow by turning supportCodegen flag off.

I can understand how badly that can read, but without help from Spark SQL devs that's all I can figure out myself. Any help appreciated.

Pozdrawiam,
Jacek Laskowski
----
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark

On Sun, Dec 10, 2017 at 10:34 PM, Stephen Boesch <javadba@gmail.com> wrote:
A relevant observation:  there was a closed/executed jira last year to remove the option to disable the codegen flag (and unsafe flag as well):   https://issues.apache.org/jira/browse/SPARK-11644

2017-12-10 13:16 GMT-08:00 Jacek Laskowski <jacek@japila.pl>:
Hi,

I'm wondering why a physical operator like GenerateExec would extend CodegenSupport [1], but had the supportCodegen flag turned off?

What's the meaning of such a combination -- be a CodegenSupport with supportCodegen off?



Pozdrawiam,
Jacek Laskowski
----
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Apache Spark 2 https://bit.ly/mastering-apache-spark