calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajat Venkatesh <rvenkat...@qubole.com>
Subject Re: Difference between Bindable and Enumerable
Date Thu, 12 Feb 2015 12:48:19 GMT
I am working on a project which is a very simple version of a cube
engine.  I dont want to execute the SQL - we have our own execution
frameworks. A clear separation between planner and executor will be helpful
in the public APIs. In the first round of reading calcite internals the
separation is not clear to me. I am not yet familiar with the code and may
very well be making bad decisions.

A high level overview of my project (to give you a data point and quick
design review) is:
The project has to:

1. Detect certain patterns in a SQL query - tables it accesses, lookups,
selective filters etc.
2. Rewrite the query if required.

I think I've finally got 1) to work. My entry point is Schemas.convert(). I
get a RelNode which I can visit and detect patterns.
I havent yet tackled 2. Hopefully will get to it in a week or two.



On Thu, Feb 12, 2015 at 12:43 PM, Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:

> Milinda> Samza's streaming query planning phase
>
> Can you clarify a bit?
> If I understand right, you aim to "wire Samza's operators in an
> optimized way and execute it as usual Samza's chain", don't you?
> In that case, it looks like you need planner only.
>
> In case you want to stream Samza's results through java.sql.ResultSet,
> then Enumerable/Bindable question makes sense.
>
> Milinda> Does this convention effect the query planning decisions other
> than
> transformations like LogicalTableScan to BindableTableScan
>
> It does not.
>
> Juilan> if people want to execute queries using just core
>
> Anybody out there wanting to execute queries using just core?
> Can we have some requirements before we implement one more executor?
>
> In 584 the discussion is regarding "being able to use Calcite core as
> planner only".
>
> Juilan> It will also make core smaller and simpler.
>
> It won't be simpler. Support of interpreter would take time.
> Let's better have separation of modules like "planner" and "executor".
> At least it would be a proof of ability to plug executor without
> changing "hard-coded rules in the planner".
>
> Juilan> Most people find it easier to implement a bindable convention
>
> Please, do not mix API vs convention.
> It does not require a new convention to process Scannable.
>
> Vladimir
>



-- 
Rajat Venkatesh | Engg Lead
Qubole Inc | www.qubole.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message