calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stamatis Zampetakis <zabe...@gmail.com>
Subject Re: ClassCastException RelOptCostImpl VolcanoCost while using HepPlanner
Date Tue, 07 May 2019 11:21:07 GMT
Thanks for looking more into this Mark.

Passing RelNodes from one planner to another is not a good idea.
Ideally this should be possible but I guess it requires CALCITE-1536 [1] to
be solved.

Another way to avoid problems like the one you observed is to copy RelNodes
from one cluster to another [2].

Best,
Stamatis

[1] https://issues.apache.org/jira/browse/CALCITE-1536
[2] https://issues.apache.org/jira/browse/CALCITE-1681


On Mon, May 6, 2019 at 12:02 PM Mark Pasterkamp <
markpasterkamp1994@gmail.com> wrote:

> I think I found the origin of the exception.
> To convert a single query to a rel node I am using the PlannerImp provided
> by Frameworks.getPlanner().
> This planner provides a VolcanoPlanner to the relnode clusters.
> When calling the HepPlanner.findBestExp() it will eventually call
> the applyTransformationResults which will compute a cost using the
> rel.getCluster().getPlanner() (which is a VolcanoPlanner).
>
>  Mark
>
> On Fri, 3 May 2019 at 19:56, Mark Pasterkamp <markpasterkamp1994@gmail.com
> >
> wrote:
>
> > I did not know this is how it works, I just copied the example above.
> > Would there be an easy way to create a RelNode containg a tablescan over
> > the materialized view "mv"?
> > Trying to create one using for instance a relbuilder gives a calcite
> > exception.
> > Otherwise I might just look for another test file in which I can get
> > access to the schema and use the materiaqlizationservice.
> >
> > Mark
> >
> > On Fri, 3 May 2019 at 18:35, Jesus Camacho Rodriguez
> > <jcamachorodriguez@cloudera.com.invalid> wrote:
> >
> >> bq . Since we are talking about materialized views, I think in most
> cases
> >> tableRel should be simply a LogicalTableScan.
> >>
> >> Stamatis is correct about this, I had not realized  tableRel == queryRel
> >> in
> >> your sample code.
> >>
> >> Thanks,
> >> Jesús
> >>
> >> On Fri, May 3, 2019 at 6:12 AM Stamatis Zampetakis <zabetak@gmail.com>
> >> wrote:
> >>
> >> > I think the main problem comes from the fact that tableRel == queryRel
> >> in
> >> > the test case you provided.
> >> > Defining the materialized view like that basically says that when you
> >> find
> >> > a part of the query that satisfies queryRel replace it with itself.
> >> > In conjunction with the rule that is used, which allows partial
> >> rewritings
> >> > using union, you end up with a rule that matches infinite number of
> >> times.
> >> > Since we are talking about materialized views, I think in most cases
> >> > tableRel should be simply a LogicalTableScan.
> >> > The idea is that expression represented by queryRel is materialized
> >> into a
> >> > table so in order to retrieve the results we only need to scan the
> >> table.
> >> >
> >> > Regarding the "if (true)" statements that you observed, most likely
> they
> >> > were introduced as release toggles [1].
> >> > However, since the last commit was in 2013 I think by now it is safe
> to
> >> > refactor that part and remove dead code.
> >> >
> >> > [1] https://www.martinfowler.com/articles/feature-toggles.html
> >> >
> >> > Best,
> >> > Stamatis
> >> >
> >> > On Fri, May 3, 2019 at 12:50 PM Mark Pasterkamp <
> >> > markpasterkamp1994@gmail.com> wrote:
> >> >
> >> > > Dear Jesus,
> >> > >
> >> > > I think your intuition in this regard is correct.
> >> > > After executing the main program in the HepPlanner the resulting
> plan
> >> > > contains a lot of circular references.
> >> > > Changing the matching order does not influence this behaviour.
> >> > >
> >> > >
> >> > > Mark
> >> > >
> >> > > On Thu, 2 May 2019 at 22:14, Jesus Camacho Rodriguez
> >> > > <jcamachorodriguez@cloudera.com.invalid> wrote:
> >> > >
> >> > > > Mark,
> >> > > >
> >> > > > I have an intuition that this happens because the rule creates
a
> >> > > partially
> >> > > > contained rewriting with a union, where one side contains a scan
> >> over
> >> > the
> >> > > > materialized view and the other side contains the query itself
> with
> >> a
> >> > > > filter on top excluding the data that is coming from the
> >> materialized
> >> > > view.
> >> > > > Then the rule is triggered on the plan representing the original
> >> query
> >> > > > again and the process is repeated. Have you tried changing the
> >> matching
> >> > > > order for your hep program?
> >> > > >
> >> > > > Thanks,
> >> > > > Jesús
> >> > > >
> >> > > > On Thu, May 2, 2019 at 8:53 AM Mark Pasterkamp <
> >> > > > markpasterkamp1994@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Stamatis,
> >> > > > >
> >> > > > > I have tried to recreate the issue but I have not been able
to
> do
> >> > > that. I
> >> > > > > was however able to create a new exception which I don't
quite
> >> > > > understand.
> >> > > > > The error happened when calcite was creating a union rewriting
> >> using
> >> > > > > materialized views. But trying to recreate this situation
gave
> me
> >> > > another
> >> > > > > interesting one.
> >> > > > > This time, the planner rewrites one of the children nodes
into
> >> > itself I
> >> > > > > would assume which causes a stack overflow. The method itself
> can
> >> be
> >> > > > found
> >> > > > > here:
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/mpasterkamp/calcite/blob/768b7928dbde5f6f9775a1119e7466d8eafafb4b/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java#L312
> >> > > > >
> >> > > > > Perhaps I am doing something wrong, perhaps not? I am not
> >> > knowledgeable
> >> > > > > enough about this to understand why this is happening. Wish
I
> >> could
> >> > > help
> >> > > > > more for that.
> >> > > > >
> >> > > > > Also, while investigating this issue I found another interesting
> >> > > artifact
> >> > > > > in de source code of the VolcanoCost. A lot of methods in
this
> >> class
> >> > > have
> >> > > > > an "if (true)"-statement like here:
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/calcite/blob/4b4d8037c5073e4eb5702b12bc4ecade31476616/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoCost.java#L100
> >> > > > >
> >> > > > > Now I was just curious, is there any reason for this to
be there
> >> that
> >> > > you
> >> > > > > know of?
> >> > > > >
> >> > > > > Thank you for responding and congratulations for your recent
> >> > > promotions.
> >> > > > >
> >> > > > >
> >> > > > > With kind regards,
> >> > > > >
> >> > > > > Mark
> >> > > > >
> >> > > > >
> >> > > > > On Thu, 2 May 2019 at 14:58, Stamatis Zampetakis <
> >> zabetak@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Said like that it looks like a bug.
> >> > > > > >
> >> > > > > > I think the best would be to reproduce the exception
as a unit
> >> test
> >> > > in
> >> > > > > > HepPlannerTest [1], RelOptRulesTest [2], or PlannerTest
[3] so
> >> that
> >> > > we
> >> > > > > > could understand better the use case.
> >> > > > > >
> >> > > > > > [1]
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java
> >> > > > > > [2]
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java
> >> > > > > > [3]
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
> >> > > > > >
> >> > > > > > On Thu, May 2, 2019 at 7:56 AM Mark Pasterkamp <
> >> > > > > > markpasterkamp1994@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > I don't, I would assume that the HepPlanner.findBestExp()
> >> > > calculates
> >> > > > > the
> >> > > > > > > cost somewhere down the line
> >> > > > > > >
> >> > > > > > > On Thu, May 2, 2019, 03:31 Yuzhao Chen <
> yuzhao.cyz@gmail.com>
> >> > > wrote:
> >> > > > > > >
> >> > > > > > > > Why you care about cost when use HepPlanner
? The
> >> HepPlanner is
> >> > > > aimed
> >> > > > > > for
> >> > > > > > > > some deterministic planning rules, we usually
do not need
> >> cost
> >> > in
> >> > > > > Hep.
> >> > > > > > > Some
> >> > > > > > > > exceptions like Join reorder may need a cost.
> >> > > > > > > >
> >> > > > > > > > What kind of planning promotion you did ?
I'm kind of
> >> curious
> >> > > about
> >> > > > > it.
> >> > > > > > > >
> >> > > > > > > > Best,
> >> > > > > > > > Danny Chan
> >> > > > > > > > 在 2019年5月1日 +0800 PM9:27,Mark Pasterkamp
<
> >> > > > > markpasterkamp1994@gmail.com
> >> > > > > > > > >,写道:
> >> > > > > > > > > Dear all,
> >> > > > > > > > >
> >> > > > > > > > > While playing around with the HepPlanner
I ran into an
> >> issue
> >> > > > where
> >> > > > > > the
> >> > > > > > > > > planner wants to rewrite a query with
a union rewrite.
> >> When
> >> > the
> >> > > > > > > > > RelMetaDataQuery computes the cost,
the cost instance
> is a
> >> > > > > > VolcanoCost.
> >> > > > > > > > > Then when it tries to calculate the
cost of one of the
> >> > union's
> >> > > > > > operands
> >> > > > > > > > it
> >> > > > > > > > > is a RelCostImpl which results in the
> ClassCastException.
> >> > > > > > > > >
> >> > > > > > > > > How would I go about solving this issue?
As far as my
> >> > knowledge
> >> > > > > > goes, I
> >> > > > > > > > am
> >> > > > > > > > > not able to change the costhandler of
the
> >> RelMetaDataQuery.
> >> > > > Another
> >> > > > > > > > > approach I could see is removing the
cast in the
> >> VolcanoCost
> >> > > > class,
> >> > > > > > > but I
> >> > > > > > > > > would hope I do not have to do that.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > With kind regards,
> >> > > > > > > > >
> >> > > > > > > > > Mark
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message