spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Chou <timchou....@gmail.com>
Subject How does Spark SQL traverse the physical tree?
Date Mon, 24 Nov 2014 17:52:21 GMT
Hi All,

I'm learning the code of Spark SQL.

I'm confused about how SchemaRDD executes each operator.

I'm tracing the code. I found toRDD() function in QueryExecution is the
start for running a query. toRDD function will run SparkPlan, which is a
tree structure.

However, I didn't find any iterative sentence in execute function for any
detail operators. It seems Spark SQL will only run the top node in this
tree.

I know the conclusion is wrong.But which code have I missed?

Thanks,
Tim

Mime
View raw message