calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stamatis Zampetakis <zabe...@gmail.com>
Subject Re: ClassCastException RelOptCostImpl VolcanoCost while using HepPlanner
Date Fri, 03 May 2019 13:11:40 GMT
I think the main problem comes from the fact that tableRel == queryRel in
the test case you provided.
Defining the materialized view like that basically says that when you find
a part of the query that satisfies queryRel replace it with itself.
In conjunction with the rule that is used, which allows partial rewritings
using union, you end up with a rule that matches infinite number of times.
Since we are talking about materialized views, I think in most cases
tableRel should be simply a LogicalTableScan.
The idea is that expression represented by queryRel is materialized into a
table so in order to retrieve the results we only need to scan the table.

Regarding the "if (true)" statements that you observed, most likely they
were introduced as release toggles [1].
However, since the last commit was in 2013 I think by now it is safe to
refactor that part and remove dead code.

[1] https://www.martinfowler.com/articles/feature-toggles.html

Best,
Stamatis

On Fri, May 3, 2019 at 12:50 PM Mark Pasterkamp <
markpasterkamp1994@gmail.com> wrote:

> Dear Jesus,
>
> I think your intuition in this regard is correct.
> After executing the main program in the HepPlanner the resulting plan
> contains a lot of circular references.
> Changing the matching order does not influence this behaviour.
>
>
> Mark
>
> On Thu, 2 May 2019 at 22:14, Jesus Camacho Rodriguez
> <jcamachorodriguez@cloudera.com.invalid> wrote:
>
> > Mark,
> >
> > I have an intuition that this happens because the rule creates a
> partially
> > contained rewriting with a union, where one side contains a scan over the
> > materialized view and the other side contains the query itself with a
> > filter on top excluding the data that is coming from the materialized
> view.
> > Then the rule is triggered on the plan representing the original query
> > again and the process is repeated. Have you tried changing the matching
> > order for your hep program?
> >
> > Thanks,
> > Jesús
> >
> > On Thu, May 2, 2019 at 8:53 AM Mark Pasterkamp <
> > markpasterkamp1994@gmail.com>
> > wrote:
> >
> > > Hi Stamatis,
> > >
> > > I have tried to recreate the issue but I have not been able to do
> that. I
> > > was however able to create a new exception which I don't quite
> > understand.
> > > The error happened when calcite was creating a union rewriting using
> > > materialized views. But trying to recreate this situation gave me
> another
> > > interesting one.
> > > This time, the planner rewrites one of the children nodes into itself I
> > > would assume which causes a stack overflow. The method itself can be
> > found
> > > here:
> > >
> > >
> > >
> >
> https://github.com/mpasterkamp/calcite/blob/768b7928dbde5f6f9775a1119e7466d8eafafb4b/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java#L312
> > >
> > > Perhaps I am doing something wrong, perhaps not? I am not knowledgeable
> > > enough about this to understand why this is happening. Wish I could
> help
> > > more for that.
> > >
> > > Also, while investigating this issue I found another interesting
> artifact
> > > in de source code of the VolcanoCost. A lot of methods in this class
> have
> > > an "if (true)"-statement like here:
> > >
> > >
> > >
> >
> https://github.com/apache/calcite/blob/4b4d8037c5073e4eb5702b12bc4ecade31476616/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoCost.java#L100
> > >
> > > Now I was just curious, is there any reason for this to be there that
> you
> > > know of?
> > >
> > > Thank you for responding and congratulations for your recent
> promotions.
> > >
> > >
> > > With kind regards,
> > >
> > > Mark
> > >
> > >
> > > On Thu, 2 May 2019 at 14:58, Stamatis Zampetakis <zabetak@gmail.com>
> > > wrote:
> > >
> > > > Said like that it looks like a bug.
> > > >
> > > > I think the best would be to reproduce the exception as a unit test
> in
> > > > HepPlannerTest [1], RelOptRulesTest [2], or PlannerTest [3] so that
> we
> > > > could understand better the use case.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java
> > > > [2]
> > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java
> > > > [3]
> > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
> > > >
> > > > On Thu, May 2, 2019 at 7:56 AM Mark Pasterkamp <
> > > > markpasterkamp1994@gmail.com>
> > > > wrote:
> > > >
> > > > > I don't, I would assume that the HepPlanner.findBestExp()
> calculates
> > > the
> > > > > cost somewhere down the line
> > > > >
> > > > > On Thu, May 2, 2019, 03:31 Yuzhao Chen <yuzhao.cyz@gmail.com>
> wrote:
> > > > >
> > > > > > Why you care about cost when use HepPlanner ? The HepPlanner
is
> > aimed
> > > > for
> > > > > > some deterministic planning rules, we usually do not need cost
in
> > > Hep.
> > > > > Some
> > > > > > exceptions like Join reorder may need a cost.
> > > > > >
> > > > > > What kind of planning promotion you did ? I'm kind of curious
> about
> > > it.
> > > > > >
> > > > > > Best,
> > > > > > Danny Chan
> > > > > > 在 2019年5月1日 +0800 PM9:27,Mark Pasterkamp <
> > > markpasterkamp1994@gmail.com
> > > > > > >,写道:
> > > > > > > Dear all,
> > > > > > >
> > > > > > > While playing around with the HepPlanner I ran into an
issue
> > where
> > > > the
> > > > > > > planner wants to rewrite a query with a union rewrite.
When the
> > > > > > > RelMetaDataQuery computes the cost, the cost instance is
a
> > > > VolcanoCost.
> > > > > > > Then when it tries to calculate the cost of one of the
union's
> > > > operands
> > > > > > it
> > > > > > > is a RelCostImpl which results in the ClassCastException.
> > > > > > >
> > > > > > > How would I go about solving this issue? As far as my knowledge
> > > > goes, I
> > > > > > am
> > > > > > > not able to change the costhandler of the RelMetaDataQuery.
> > Another
> > > > > > > approach I could see is removing the cast in the VolcanoCost
> > class,
> > > > > but I
> > > > > > > would hope I do not have to do that.
> > > > > > >
> > > > > > >
> > > > > > > With kind regards,
> > > > > > >
> > > > > > > Mark
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message