calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Pasterkamp <markpasterkamp1...@gmail.com>
Subject Re: ClassCastException RelOptCostImpl VolcanoCost while using HepPlanner
Date Mon, 06 May 2019 10:01:41 GMT
I think I found the origin of the exception.
To convert a single query to a rel node I am using the PlannerImp provided
by Frameworks.getPlanner().
This planner provides a VolcanoPlanner to the relnode clusters.
When calling the HepPlanner.findBestExp() it will eventually call
the applyTransformationResults which will compute a cost using the
rel.getCluster().getPlanner() (which is a VolcanoPlanner).

 Mark

On Fri, 3 May 2019 at 19:56, Mark Pasterkamp <markpasterkamp1994@gmail.com>
wrote:

> I did not know this is how it works, I just copied the example above.
> Would there be an easy way to create a RelNode containg a tablescan over
> the materialized view "mv"?
> Trying to create one using for instance a relbuilder gives a calcite
> exception.
> Otherwise I might just look for another test file in which I can get
> access to the schema and use the materiaqlizationservice.
>
> Mark
>
> On Fri, 3 May 2019 at 18:35, Jesus Camacho Rodriguez
> <jcamachorodriguez@cloudera.com.invalid> wrote:
>
>> bq . Since we are talking about materialized views, I think in most cases
>> tableRel should be simply a LogicalTableScan.
>>
>> Stamatis is correct about this, I had not realized  tableRel == queryRel
>> in
>> your sample code.
>>
>> Thanks,
>> Jesús
>>
>> On Fri, May 3, 2019 at 6:12 AM Stamatis Zampetakis <zabetak@gmail.com>
>> wrote:
>>
>> > I think the main problem comes from the fact that tableRel == queryRel
>> in
>> > the test case you provided.
>> > Defining the materialized view like that basically says that when you
>> find
>> > a part of the query that satisfies queryRel replace it with itself.
>> > In conjunction with the rule that is used, which allows partial
>> rewritings
>> > using union, you end up with a rule that matches infinite number of
>> times.
>> > Since we are talking about materialized views, I think in most cases
>> > tableRel should be simply a LogicalTableScan.
>> > The idea is that expression represented by queryRel is materialized
>> into a
>> > table so in order to retrieve the results we only need to scan the
>> table.
>> >
>> > Regarding the "if (true)" statements that you observed, most likely they
>> > were introduced as release toggles [1].
>> > However, since the last commit was in 2013 I think by now it is safe to
>> > refactor that part and remove dead code.
>> >
>> > [1] https://www.martinfowler.com/articles/feature-toggles.html
>> >
>> > Best,
>> > Stamatis
>> >
>> > On Fri, May 3, 2019 at 12:50 PM Mark Pasterkamp <
>> > markpasterkamp1994@gmail.com> wrote:
>> >
>> > > Dear Jesus,
>> > >
>> > > I think your intuition in this regard is correct.
>> > > After executing the main program in the HepPlanner the resulting plan
>> > > contains a lot of circular references.
>> > > Changing the matching order does not influence this behaviour.
>> > >
>> > >
>> > > Mark
>> > >
>> > > On Thu, 2 May 2019 at 22:14, Jesus Camacho Rodriguez
>> > > <jcamachorodriguez@cloudera.com.invalid> wrote:
>> > >
>> > > > Mark,
>> > > >
>> > > > I have an intuition that this happens because the rule creates a
>> > > partially
>> > > > contained rewriting with a union, where one side contains a scan
>> over
>> > the
>> > > > materialized view and the other side contains the query itself with
>> a
>> > > > filter on top excluding the data that is coming from the
>> materialized
>> > > view.
>> > > > Then the rule is triggered on the plan representing the original
>> query
>> > > > again and the process is repeated. Have you tried changing the
>> matching
>> > > > order for your hep program?
>> > > >
>> > > > Thanks,
>> > > > Jesús
>> > > >
>> > > > On Thu, May 2, 2019 at 8:53 AM Mark Pasterkamp <
>> > > > markpasterkamp1994@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi Stamatis,
>> > > > >
>> > > > > I have tried to recreate the issue but I have not been able to
do
>> > > that. I
>> > > > > was however able to create a new exception which I don't quite
>> > > > understand.
>> > > > > The error happened when calcite was creating a union rewriting
>> using
>> > > > > materialized views. But trying to recreate this situation gave
me
>> > > another
>> > > > > interesting one.
>> > > > > This time, the planner rewrites one of the children nodes into
>> > itself I
>> > > > > would assume which causes a stack overflow. The method itself
can
>> be
>> > > > found
>> > > > > here:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/mpasterkamp/calcite/blob/768b7928dbde5f6f9775a1119e7466d8eafafb4b/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java#L312
>> > > > >
>> > > > > Perhaps I am doing something wrong, perhaps not? I am not
>> > knowledgeable
>> > > > > enough about this to understand why this is happening. Wish I
>> could
>> > > help
>> > > > > more for that.
>> > > > >
>> > > > > Also, while investigating this issue I found another interesting
>> > > artifact
>> > > > > in de source code of the VolcanoCost. A lot of methods in this
>> class
>> > > have
>> > > > > an "if (true)"-statement like here:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/calcite/blob/4b4d8037c5073e4eb5702b12bc4ecade31476616/core/src/main/java/org/apache/calcite/plan/volcano/VolcanoCost.java#L100
>> > > > >
>> > > > > Now I was just curious, is there any reason for this to be there
>> that
>> > > you
>> > > > > know of?
>> > > > >
>> > > > > Thank you for responding and congratulations for your recent
>> > > promotions.
>> > > > >
>> > > > >
>> > > > > With kind regards,
>> > > > >
>> > > > > Mark
>> > > > >
>> > > > >
>> > > > > On Thu, 2 May 2019 at 14:58, Stamatis Zampetakis <
>> zabetak@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Said like that it looks like a bug.
>> > > > > >
>> > > > > > I think the best would be to reproduce the exception as
a unit
>> test
>> > > in
>> > > > > > HepPlannerTest [1], RelOptRulesTest [2], or PlannerTest
[3] so
>> that
>> > > we
>> > > > > > could understand better the use case.
>> > > > > >
>> > > > > > [1]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/HepPlannerTest.java
>> > > > > > [2]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/test/RelOptRulesTest.java
>> > > > > > [3]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/calcite/blob/master/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
>> > > > > >
>> > > > > > On Thu, May 2, 2019 at 7:56 AM Mark Pasterkamp <
>> > > > > > markpasterkamp1994@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > I don't, I would assume that the HepPlanner.findBestExp()
>> > > calculates
>> > > > > the
>> > > > > > > cost somewhere down the line
>> > > > > > >
>> > > > > > > On Thu, May 2, 2019, 03:31 Yuzhao Chen <yuzhao.cyz@gmail.com>
>> > > wrote:
>> > > > > > >
>> > > > > > > > Why you care about cost when use HepPlanner ?
The
>> HepPlanner is
>> > > > aimed
>> > > > > > for
>> > > > > > > > some deterministic planning rules, we usually
do not need
>> cost
>> > in
>> > > > > Hep.
>> > > > > > > Some
>> > > > > > > > exceptions like Join reorder may need a cost.
>> > > > > > > >
>> > > > > > > > What kind of planning promotion you did ? I'm
kind of
>> curious
>> > > about
>> > > > > it.
>> > > > > > > >
>> > > > > > > > Best,
>> > > > > > > > Danny Chan
>> > > > > > > > 在 2019年5月1日 +0800 PM9:27,Mark Pasterkamp
<
>> > > > > markpasterkamp1994@gmail.com
>> > > > > > > > >,写道:
>> > > > > > > > > Dear all,
>> > > > > > > > >
>> > > > > > > > > While playing around with the HepPlanner
I ran into an
>> issue
>> > > > where
>> > > > > > the
>> > > > > > > > > planner wants to rewrite a query with a union
rewrite.
>> When
>> > the
>> > > > > > > > > RelMetaDataQuery computes the cost, the cost
instance is a
>> > > > > > VolcanoCost.
>> > > > > > > > > Then when it tries to calculate the cost
of one of the
>> > union's
>> > > > > > operands
>> > > > > > > > it
>> > > > > > > > > is a RelCostImpl which results in the ClassCastException.
>> > > > > > > > >
>> > > > > > > > > How would I go about solving this issue?
As far as my
>> > knowledge
>> > > > > > goes, I
>> > > > > > > > am
>> > > > > > > > > not able to change the costhandler of the
>> RelMetaDataQuery.
>> > > > Another
>> > > > > > > > > approach I could see is removing the cast
in the
>> VolcanoCost
>> > > > class,
>> > > > > > > but I
>> > > > > > > > > would hope I do not have to do that.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > With kind regards,
>> > > > > > > > >
>> > > > > > > > > Mark
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message