spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject InMemoryTableScanExec.inputRDD and buffers (RDD[CachedBatch])
Date Mon, 14 May 2018 11:30:32 GMT
Hi,

Is there any reason why InMemoryTableScanExec.inputRDD does not use the
buffers local value [1] for the non-batch case [2]? Just curious as I ran
into it and thought I'd do a tiny refactoring.

[1]
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala?utf8=%E2%9C%93#L105
[2]
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryTableScanExec.scala?utf8=%E2%9C%93#L125

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski

Mime
View raw message