calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessandro Solimando <alessandro.solima...@gmail.com>
Subject copy method in RelSubset class
Date Sun, 11 Feb 2018 21:18:45 GMT
Hello community,

I am adding a SparkAdapter test with the following query:

> select *
> from *(values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)*
> where x between 3 and 4
>
> When executed, an exception is thrown (the full stack trace is at the end
of the email) in *copy* method in *RelSubset* class, while Calcite is
trying to get rid of unused terms (specifically, *trimUnusedFields* method
from *SqlToRelConverted* class).

The signature of copy is as follows: public RelNode copy(RelTraitSet
traitSet, List<RelNode> inputs)

First of all, I don't understand the reason for the
*UnsupportedOperationException* in the first place. Why a RelSubset
shouldn't be copied?

Assuming that the functionality is simply missing, I have considered two
alternatives for implementing it:
1) copy as the identity function -> all Calcite tests pass, but I am
ignoring the *traitSet* parameter in this way, looks odd
2) I have tried to build a new *RelSubset* by reusing the cluster and set
information from the object, and the trait argument of copy -> assert
traits.allSimple();
fails in the constructor

In my example, the trait "[1]" (ordering detected at tuple level on the 2nd
component) is transformed into a composite trait "[[1]]", this makes the
assertion fail.
While I know what a trait is, I don't understand what a composite one is.
Do you have a concrete example?

So the problem here is the introduction of the composite trait, which is
caused by the *replace* method in *trimUnusedFields*:

if (isTrimUnusedFields()) {
>>   final RelFieldTrimmer trimmer = newFieldTrimmer();
>>   final List<RelCollation> collations =
>>     rootRel.getTraitSet().getTraits(RelCollationTraitDef.INSTANCE);
>>   rootRel = trimmer.trim(rootRel);
>>   if (!ordered
>>   && collations != null
>>   && !collations.isEmpty()
>>   && !collations.equals(ImmutableList.of(RelCollations.EMPTY))) {
>>     final RelTraitSet traitSet = rootRel.getTraitSet()
>>       .replace(RelCollationTraitDef.INSTANCE, collations);
>>     rootRel = rootRel.copy(traitSet, rootRel.getInputs());
>>   }
>>   if (SQL2REL_LOGGER.isDebugEnabled()) {
>>     SQL2REL_LOGGER.debug(
>>       RelOptUtil.dumpPlan("Plan after trimming unused fields", rootRel,
>>       SqlExplainFormat.TEXT, SqlExplainLevel.EXPPLAN_ATTRIBUTES));
>>    }
>> }
>
>
It is also not clear to me what the first *if* is trying to accomplish
here. I mean, the traits are never modified here, so it really looks like
the only reason for calling *replace* is to apply whatever side effect this
method has (the conversion from traits to composite traits looks the only
one to me), and in the specific situations matching the if condition. Can
you clarify which scenario the if is handling?

I would appreciate also a feedback on the implementation of copy as
identity. Is it correct for you?
Or do you suggest the second option by enforcing a flattening of traits
before calling the constructor?

Best regards,
Alessandro

The full stack trace:

java.lang.RuntimeException: With materializationsEnabled=false, limit=0
>
> at
>> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
>
> at
>> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
>
> at
>> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
>
> at
>> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
>
> at
>> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
>
> at
>> org.apache.calcite.test.SparkAdapterTest.testFilterBetween(SparkAdapterTest.java:460)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>
> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>
> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>
> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>
> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>
> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>
> at
>> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
>
> at
>> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
>
> at
>> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
>
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
>
> Caused by: java.sql.SQLException: Error while executing SQL "select *
>
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
>
> where x between 3 and 4": null
>
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
>
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
>
> at
>> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
>
> at
>> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
>
> at
>> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
>
> ... 27 more
>
> Caused by: java.lang.UnsupportedOperationException
>
> at org.apache.calcite.plan.volcano.RelSubset.copy(RelSubset.java:149)
>
> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:517)
>
> at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:391)
>
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:304)
>
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
>
> at
>> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781)
>
> at
>> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
>
> at
>> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
>
> at
>> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
>
> at
>> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
>
> at
>> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
>
> at
>> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
>
> ... 29 more
>
>
>> java.lang.RuntimeException: exception while executing [select *
>
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
>
> where x between 3 and 4]
>
>
>> at
>> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1351)
>
> at
>> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1329)
>
> at
>> org.apache.calcite.test.CalciteAssert$AssertQuery.returnsUnordered(CalciteAssert.java:1357)
>
> at
>> org.apache.calcite.test.SparkAdapterTest.commonTester(SparkAdapterTest.java:93)
>
> at
>> org.apache.calcite.test.SparkAdapterTest.testFilterBetween(SparkAdapterTest.java:460)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>
> at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>
> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>
> at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>
> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>
> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>
> at
>> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
>
> at
>> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
>
> at
>> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
>
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
>
> Caused by: java.lang.RuntimeException: With materializationsEnabled=false,
>> limit=0
>
> at
>> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:600)
>
> at
>> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1346)
>
> ... 26 more
>
> Caused by: java.sql.SQLException: Error while executing SQL "select *
>
> from (values (1, 'a'), (2, 'b'), (3, 'b'), (4, 'c'), (2, 'c')) as t(x, y)
>
> where x between 3 and 4": null
>
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
>
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
>
> at
>> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
>
> at
>> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
>
> at
>> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
>
> ... 27 more
>
> Caused by: java.lang.UnsupportedOperationException
>
> at org.apache.calcite.plan.volcano.RelSubset.copy(RelSubset.java:149)
>
> at
>> org.apache.calcite.sql2rel.SqlToRelConverter.trimUnusedFields(SqlToRelConverter.java:517)
>
> at org.apache.calcite.prepare.Prepare.trimUnusedFields(Prepare.java:391)
>
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:304)
>
> at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
>
> at
>> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781)
>
> at
>> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
>
> at
>> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
>
> at
>> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
>
> at
>> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
>
> at
>> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
>
> at
>> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
>
> ... 29 more
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message