calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammad Gelbana <m.gelb...@gmail.com>
Subject Re: Difficulties with implementing a table for a custom convention
Date Mon, 06 May 2019 15:13:29 GMT
Thank you all for your time but unfortunately I'm unable to overcome this
challenge so far. Allow me to explain more.

*This is how I'm using calcite*
Planner planner = Frameworks.getPlanner(frameworkConfig); //
frameworkConfig is irrelevant I suppose
SqlNode parsed = planner.parse("SELECT Tenant FROM Audit.audit where Tenant
!= 'castle'");
SqlNode validated = planner.validate(parsed);
RelRoot root = planner.rel(validated);
PreparedStatement prepared = RelRunners.run(root.rel);
ResultSet rs = prepared.executeQuery());

*This is the final physical plan I'm achieving*
EnumerableInterpreter: rowcount = 50.0, cumulative cost = {25.0 rows, 25.0
cpu, 0.0 io}, id = 102
  BindableProject(Tenant=[$2]): rowcount = 50.0, cumulative cost = {160.0
rows, 166.0 cpu, 0.0 io}, id = 100
    GelbanaBindableRel: rowcount = 50.0, cumulative cost = {110.0 rows,
116.0 cpu, 0.0 io}, id = 98 *// My converter node*
      GelbanaFilter(condition=[<>($2, 'demo')]): rowcount = 50.0,
cumulative cost = {105.0 rows, 111.0 cpu, 0.0 io}, id = 96
        GelbanaQuery(table=[[Audit, audit]]): rowcount = 100.0, cumulative
cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 3 *// My table scan*

My problem is that the interpreter goes through all the plan's operators
[1] and for each operator it attempts to wrap it as an instance of
org.apache.calcite.interpreter.Node [2] so the interpreter would call the
Node's method "run" [3] for each wrapped operator when it starts executing
the plan. But if it couldn't wrap the operator as a Node [4], it checks if
the operator implements the InterpretableRel interface to call it's
"implement" method. But if it doesn't implement the InterpretableRel
interface in that case, an exception is thrown.

My issue is that I don't need my Gelbana operators, including the table
scan which is this thread is all about, to be wrapped as nodes neither
implement the InterpretableRel interface because my converter
(GelbanaBindableRel) will be responsible for running its inputs, including
eventually the table scan(s).

To fix that, I did the following for my converter
@Override
public void childrenAccept(RelVisitor visitor) {
    if (visitor instanceof CoreCompiler) return;
}

This allows me to prevent the interpreter from visiting my converter's
inputs [1] to avoid the trouble mentioned above. But it feels like I'm
hacking my way through calcite and not using it the way its desgined to. Or
is this a correct approach ? Could this have any side effects I might worry
about ?

[1]
https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L447
[2]
https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L451
[3]
https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L130
[4]
https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L452

Thanks,
Gelbana


On Sun, May 5, 2019 at 11:43 PM Stamatis Zampetakis <zabetak@gmail.com>
wrote:

> Hi Muhammad,
>
> I'm not sure why you need to implement InterpretableRel interface. I'm
> probably missing some details.
>
> I suppose you are using Calcite through the JDBC interface thus I guess you
> are relying on CalcitePrepareImpl.
> If that's the case then using TranslatableTable interface seems like a good
> idea.
> The latter means that you are using the default VolcanoPlanner that expects
> the final result to be in Bindable or Enumerable convention [2].
> You may define various operators and tables to be in CustomConvention but
> for sure you need a Converter
> and respective rules to go from CustomConvention to EnumerableConvention
> (similar to MongoToEnumerableConverter and MongoToEnumerableConverterRule).
>
> The final plan should look be similar to the one below:
>
> CustomToEnumerableConverter
>   CustomProject(product_id=[CAST(ITEM($0, 'product_id')):DOUBLE])
>     CustomTableScan(table=[[_foodmart, sales_fact_1998]])"
>
> Best,
> Stamatis
>
> [2]
>
> https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java#L719
>
> On Sun, May 5, 2019 at 4:13 AM Yuzhao Chen <yuzhao.cyz@gmail.com> wrote:
>
> > You can set up the target traits to include the convention you want when
> > invoke the Program#run method [1], with the converters as rules of the
> > program.
> >
> > [1]
> >
> https://github.com/apache/calcite/blob/9ece70f5dcdb00dbc6712496c51f52c05178d4aa/core/src/main/java/org/apache/calcite/tools/Program.java#L38
> >
> > Best,
> > Danny Chan
> > 在 2019年5月4日 +0800 PM6:48,Muhammad Gelbana <m.gelbana@gmail.com>,写道:
> > > I implemented a new convention but I'm facing difficulties with
> > > implementing tables for that convention.
> > >
> > > Since I need to apply my convention rules and converters, I assume I
> must
> > > implement TranslatableTable so I can override the RelNode produced by
> the
> > > toRel method and provide the rules I need through the
> > > RelNode.register(RelOptPlanner planner) call back (Is there another way
> > ?)
> > >
> > > At the same time, I need for my table's RelNode to implement my
> > > convention's interface.
> > >
> > > After implementing TranslatableTable and my convention's interface, I
> > > discovered that my table's RelNode have to either implement an
> > interpreter
> > > node or the InterpretableRel interface. But this led to what seems to
> me
> > as
> > > that my table scan is executed twice.
> > >
> > > I tried many things, such as having my table extend TableScan,
> > > AbstractEnumerable2 QueryableTable and other interfaces/class that
> either
> > > didn't give me what I want or failed with strange exceptions.
> > >
> > > What I need is to implement my table scan following the same convention
> > I'm
> > > writing. So what is the correct way to do this ?
> > >
> > > The MongoDB adapter way is to implement the TranslatableTable and the
> > > QueryableTable interfaces, but that's confusing me. I mean, it's
> already
> > > implementing its own new convention interface, why would it need to
> > > implement another one to implement its table scans ? I thought while
> > > executing the final plan, the first calcite identified node (enumerable
> > or
> > > bindable, enumerable as in MongoDB adapter's example), the interpreter
> > > would execute that node, which will internally execute all its input
> > nodes
> > > including the table scan, correct ?
> > >
> > > Thanks,
> > > Gelbana
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message