calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesus Camacho Rodriguez <jcamachorodrig...@cloudera.com.INVALID>
Subject Re: ClassCastException RelOptCostImpl VolcanoCost while using HepPlanner
Date Fri, 03 May 2019 16:35:11 GMT
bq . Since we are talking about materialized views, I think in most cases
tableRel should be simply a LogicalTableScan.

Stamatis is correct about this, I had not realized  tableRel == queryRel in
your sample code.

Thanks,
Jesús

On Fri, May 3, 2019 at 6:12 AM Stamatis Zampetakis <zabetak@gmail.com>
wrote:

> I think the main problem comes from the fact that tableRel == queryRel in
> the test case you provided.
> Defining the materialized view like that basically says that when you find
> a part of the query that satisfies queryRel replace it with itself.
> In conjunction with the rule that is used, which allows partial rewritings
> using union, you end up with a rule that matches infinite number of times.
> Since we are talking about materialized views, I think in most cases
> tableRel should be simply a LogicalTableScan.
> The idea is that expression represented by queryRel is materialized into a
> table so in order to retrieve the results we only need to scan the table.
>
> Regarding the "if (true)" statements that you observed, most likely they
> were introduced as release toggles [1].
> However, since the last commit was in 2013 I think by now it is safe to
> refactor that part and remove dead code.
>
> [1] https://www.martinfowler.com/articles/feature-toggles.html
>
> Best,
> Stamatis
>
> On Fri, May 3, 2019 at 12:50 PM Mark Pasterkamp <
> markpasterkamp1994@gmail.com> wrote:
>
> > Dear Jesus,
> >
> > I think your intuition in this regard is correct.
> > After executing the main program in the HepPlanner the resulting plan
> > contains a lot of circular references.
> > Changing the matching order does not influence this behaviour.
> >
> >
> > Mark
> >
> > On Thu, 2 May 2019 at 22:14, Jesus Camacho Rodriguez
> > <jcamachorodriguez@cloudera.com.invalid> wrote:
> >
> > > Mark,
> > >
> > > I have an intuition that this happens because the rule creates a
> > partially
> > > contained rewriting with a union, where one side contains a scan over
> the
> > > materialized view and the other side contains the query itself with a
> > > filter on top excluding the data that is coming from the materialized
> > view.
> > > Then the rule is triggered on the plan representing the original query
> > > again and the process is repeated. Have you tried changing the matching
> > > order for your hep program?
> > >
> > > Thanks,
> > > Jesús
> > >
> > > On Thu, May 2, 2019 at 8:53 AM Mark Pasterkamp <
> > > markpasterkamp1994@gmail.com>
> > > wrote:
> > >
> > > > Hi Stamatis,
> > > >
> > > > I have tried to recreate the issue but I have not been able to do
> > that. I
> > > > was however able to create a new exception which I don't quite
> > > understand.
> > > > The error happened when calcite was creating a union rewriting using
> > > > materialized views. But trying to recreate this situation gave me
> > another
> > > > interesting one.
> > > > This time, the planner rewrites one of the children nodes into
> itself I
> > > > would assume which causes a stack overflow. The method itself can be
> > > found
> > > > here:
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/mpasterkamp/calcite/blob/768b7928dbde5f6f9775a1119e7466d8eafafb4b/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java#L312
> > > >
> > > > Perhaps I am doing something wrong, perhaps not? I am not
> knowledgeable
> > > > enough about this to understand why this is happening. Wish I could
> > help
> > > > more for that.
> > > >
> > > > Also, while investigating this issue I found another interesting
> > artifact
> > > > in de source code of the VolcanoCost. A lot of methods in this class
> > have
> > > > an "if (true)"-statement like here:
> > > >
> > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/4b4d8037c5073e4eb5702b12bc4ecade31476616/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoCost.java#L100
> > > >
> > > > Now I was just curious, is there any reason for this to be there that
> > you
> > > > know of?
> > > >
> > > > Thank you for responding and congratulations for your recent
> > promotions.
> > > >
> > > >
> > > > With kind regards,
> > > >
> > > > Mark
> > > >
> > > >
> > > > On Thu, 2 May 2019 at 14:58, Stamatis Zampetakis <zabetak@gmail.com>
> > > > wrote:
> > > >
> > > > > Said like that it looks like a bug.
> > > > >
> > > > > I think the best would be to reproduce the exception as a unit test
> > in
> > > > > HepPlannerTest [1], RelOptRulesTest [2], or PlannerTest [3] so that
> > we
> > > > > could understand better the use case.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java
> > > > > [3]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
> > > > >
> > > > > On Thu, May 2, 2019 at 7:56 AM Mark Pasterkamp <
> > > > > markpasterkamp1994@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > I don't, I would assume that the HepPlanner.findBestExp()
> > calculates
> > > > the
> > > > > > cost somewhere down the line
> > > > > >
> > > > > > On Thu, May 2, 2019, 03:31 Yuzhao Chen <yuzhao.cyz@gmail.com>
> > wrote:
> > > > > >
> > > > > > > Why you care about cost when use HepPlanner ? The HepPlanner
is
> > > aimed
> > > > > for
> > > > > > > some deterministic planning rules, we usually do not need
cost
> in
> > > > Hep.
> > > > > > Some
> > > > > > > exceptions like Join reorder may need a cost.
> > > > > > >
> > > > > > > What kind of planning promotion you did ? I'm kind of curious
> > about
> > > > it.
> > > > > > >
> > > > > > > Best,
> > > > > > > Danny Chan
> > > > > > > 在 2019年5月1日 +0800 PM9:27,Mark Pasterkamp <
> > > > markpasterkamp1994@gmail.com
> > > > > > > >,写道:
> > > > > > > > Dear all,
> > > > > > > >
> > > > > > > > While playing around with the HepPlanner I ran into
an issue
> > > where
> > > > > the
> > > > > > > > planner wants to rewrite a query with a union rewrite.
When
> the
> > > > > > > > RelMetaDataQuery computes the cost, the cost instance
is a
> > > > > VolcanoCost.
> > > > > > > > Then when it tries to calculate the cost of one of
the
> union's
> > > > > operands
> > > > > > > it
> > > > > > > > is a RelCostImpl which results in the ClassCastException.
> > > > > > > >
> > > > > > > > How would I go about solving this issue? As far as
my
> knowledge
> > > > > goes, I
> > > > > > > am
> > > > > > > > not able to change the costhandler of the RelMetaDataQuery.
> > > Another
> > > > > > > > approach I could see is removing the cast in the VolcanoCost
> > > class,
> > > > > > but I
> > > > > > > > would hope I do not have to do that.
> > > > > > > >
> > > > > > > >
> > > > > > > > With kind regards,
> > > > > > > >
> > > > > > > > Mark
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message