calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Execute Calcite logical plans on Spark
Date Sat, 22 Apr 2017 04:37:46 GMT
Since the SQL will pass through Spark’s Catalyst SQL optimizer, I suspect that most of the
optimizations Calcite has made will be negated.

The ideal thing would be to translate the Calcite plan (RelNodes) into a Spark program, which
would then be executed without further optimization. Calcite’s Spark adapter does that (but
is old and incomplete); I heard rumors of an improved Spark adapter but it hasn’t landed
in open source yet, and I don’t know when/whether that will happen.

Julian


> On Apr 21, 2017, at 3:50 PM, Khai Tran <khtran@linkedin.com.INVALID> wrote:
> 
> Hi all,
> I'm trying to figure out some way to execute a given Calcite logical plan
> on Spark. One possible way to convert the Calcite plan back to SQL and run
> it as Spark SQL.
> 
> Any ideas on that?
> 
> Thanks,
> Khai


Mime
View raw message