calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stamatis Zampetakis <zabe...@gmail.com>
Subject Re: Difficulties with implementing a table for a custom convention
Date Tue, 07 May 2019 11:07:39 GMT
The plan seems reasonable to me so as you said there seems to be a problem
with the Interpreter.
Overriding childrenAccept seems like a good workaround but I think it would
be nice to fix the Interpreter.
Can you please log a JIRA issue for this?

As a general remark, I don't use Bindable convention very often since it is
not very well tested.
Although Enumerable is a bit more difficult to understand it is more
stable. Maybe it would be
a good idea to change your converter from Bindable to Enumerable.

Best,
Stamatis

On Mon, May 6, 2019 at 5:13 PM Muhammad Gelbana <m.gelbana@gmail.com> wrote:

> Thank you all for your time but unfortunately I'm unable to overcome this
> challenge so far. Allow me to explain more.
>
> *This is how I'm using calcite*
> Planner planner = Frameworks.getPlanner(frameworkConfig); //
> frameworkConfig is irrelevant I suppose
> SqlNode parsed = planner.parse("SELECT Tenant FROM Audit.audit where Tenant
> != 'castle'");
> SqlNode validated = planner.validate(parsed);
> RelRoot root = planner.rel(validated);
> PreparedStatement prepared = RelRunners.run(root.rel);
> ResultSet rs = prepared.executeQuery());
>
> *This is the final physical plan I'm achieving*
> EnumerableInterpreter: rowcount = 50.0, cumulative cost = {25.0 rows, 25.0
> cpu, 0.0 io}, id = 102
>   BindableProject(Tenant=[$2]): rowcount = 50.0, cumulative cost = {160.0
> rows, 166.0 cpu, 0.0 io}, id = 100
>     GelbanaBindableRel: rowcount = 50.0, cumulative cost = {110.0 rows,
> 116.0 cpu, 0.0 io}, id = 98 *// My converter node*
>       GelbanaFilter(condition=[<>($2, 'demo')]): rowcount = 50.0,
> cumulative cost = {105.0 rows, 111.0 cpu, 0.0 io}, id = 96
>         GelbanaQuery(table=[[Audit, audit]]): rowcount = 100.0, cumulative
> cost = {100.0 rows, 101.0 cpu, 0.0 io}, id = 3 *// My table scan*
>
> My problem is that the interpreter goes through all the plan's operators
> [1] and for each operator it attempts to wrap it as an instance of
> org.apache.calcite.interpreter.Node [2] so the interpreter would call the
> Node's method "run" [3] for each wrapped operator when it starts executing
> the plan. But if it couldn't wrap the operator as a Node [4], it checks if
> the operator implements the InterpretableRel interface to call it's
> "implement" method. But if it doesn't implement the InterpretableRel
> interface in that case, an exception is thrown.
>
> My issue is that I don't need my Gelbana operators, including the table
> scan which is this thread is all about, to be wrapped as nodes neither
> implement the InterpretableRel interface because my converter
> (GelbanaBindableRel) will be responsible for running its inputs, including
> eventually the table scan(s).
>
> To fix that, I did the following for my converter
> @Override
> public void childrenAccept(RelVisitor visitor) {
>     if (visitor instanceof CoreCompiler) return;
> }
>
> This allows me to prevent the interpreter from visiting my converter's
> inputs [1] to avoid the trouble mentioned above. But it feels like I'm
> hacking my way through calcite and not using it the way its desgined to. Or
> is this a correct approach ? Could this have any side effects I might worry
> about ?
>
> [1]
>
> https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L447
> [2]
>
> https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L451
> [3]
>
> https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L130
> [4]
>
> https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/interpreter/Interpreter.java#L452
>
> Thanks,
> Gelbana
>
>
> On Sun, May 5, 2019 at 11:43 PM Stamatis Zampetakis <zabetak@gmail.com>
> wrote:
>
> > Hi Muhammad,
> >
> > I'm not sure why you need to implement InterpretableRel interface. I'm
> > probably missing some details.
> >
> > I suppose you are using Calcite through the JDBC interface thus I guess
> you
> > are relying on CalcitePrepareImpl.
> > If that's the case then using TranslatableTable interface seems like a
> good
> > idea.
> > The latter means that you are using the default VolcanoPlanner that
> expects
> > the final result to be in Bindable or Enumerable convention [2].
> > You may define various operators and tables to be in CustomConvention but
> > for sure you need a Converter
> > and respective rules to go from CustomConvention to EnumerableConvention
> > (similar to MongoToEnumerableConverter and
> MongoToEnumerableConverterRule).
> >
> > The final plan should look be similar to the one below:
> >
> > CustomToEnumerableConverter
> >   CustomProject(product_id=[CAST(ITEM($0, 'product_id')):DOUBLE])
> >     CustomTableScan(table=[[_foodmart, sales_fact_1998]])"
> >
> > Best,
> > Stamatis
> >
> > [2]
> >
> >
> https://github.com/apache/calcite/blob/0d504d20d47542e8d461982512ae0e7a94e4d6cb/core/src/main/java/org/apache/calcite/prepare/CalcitePrepareImpl.java#L719
> >
> > On Sun, May 5, 2019 at 4:13 AM Yuzhao Chen <yuzhao.cyz@gmail.com> wrote:
> >
> > > You can set up the target traits to include the convention you want
> when
> > > invoke the Program#run method [1], with the converters as rules of the
> > > program.
> > >
> > > [1]
> > >
> >
> https://github.com/apache/calcite/blob/9ece70f5dcdb00dbc6712496c51f52c05178d4aa/core/src/main/java/org/apache/calcite/tools/Program.java#L38
> > >
> > > Best,
> > > Danny Chan
> > > 在 2019年5月4日 +0800 PM6:48,Muhammad Gelbana <m.gelbana@gmail.com>,写道:
> > > > I implemented a new convention but I'm facing difficulties with
> > > > implementing tables for that convention.
> > > >
> > > > Since I need to apply my convention rules and converters, I assume I
> > must
> > > > implement TranslatableTable so I can override the RelNode produced by
> > the
> > > > toRel method and provide the rules I need through the
> > > > RelNode.register(RelOptPlanner planner) call back (Is there another
> way
> > > ?)
> > > >
> > > > At the same time, I need for my table's RelNode to implement my
> > > > convention's interface.
> > > >
> > > > After implementing TranslatableTable and my convention's interface, I
> > > > discovered that my table's RelNode have to either implement an
> > > interpreter
> > > > node or the InterpretableRel interface. But this led to what seems to
> > me
> > > as
> > > > that my table scan is executed twice.
> > > >
> > > > I tried many things, such as having my table extend TableScan,
> > > > AbstractEnumerable2 QueryableTable and other interfaces/class that
> > either
> > > > didn't give me what I want or failed with strange exceptions.
> > > >
> > > > What I need is to implement my table scan following the same
> convention
> > > I'm
> > > > writing. So what is the correct way to do this ?
> > > >
> > > > The MongoDB adapter way is to implement the TranslatableTable and the
> > > > QueryableTable interfaces, but that's confusing me. I mean, it's
> > already
> > > > implementing its own new convention interface, why would it need to
> > > > implement another one to implement its table scans ? I thought while
> > > > executing the final plan, the first calcite identified node
> (enumerable
> > > or
> > > > bindable, enumerable as in MongoDB adapter's example), the
> interpreter
> > > > would execute that node, which will internally execute all its input
> > > nodes
> > > > including the table scan, correct ?
> > > >
> > > > Thanks,
> > > > Gelbana
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message