calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Adding Exchange operator and Distribution trait
Date Fri, 13 Feb 2015 08:39:15 GMT
I have committed, on my branch
https://github.com/julianhyde/incubator-calcite/tree/calcite-594, a fix for
https://issues.apache.org/jira/browse/CALCITE-594 that I think meets
Drill's and Ashutosh's requirements. Please review it before I commit to
master.

I have 3 questions:

1. Can we please shorten the distribution type name HASH_DISTRIBUTED to
HASH (similarly RANDOM_DISTRIBUTED etc.)?

2. Drill has DrillDistributionTrait.DEFAULT, but I'd rather not create
RelDistributions.DEFAULT. (The default will be different for different
systems using Calcite.)

3. RelDistributionTraitDef.convert does not create specific physical
operators as it does in Drill, because there are no physical
implementations of Exchange, just LogicalExchange. Drill will need to
create rules to convert LogicalExchange to HashToRandomExchangePrel, etc.
Hope that is OK.

Julian



On Feb 12, 2015, at 1:57 PM, Jacques Nadeau <jacques@apache.org> wrote:

Got it.

On Thu, Feb 12, 2015 at 11:44 AM, Julian Hyde <julian@hydromatic.net> wrote:

RelMdDistribution (the "Md" that distinguishes it from
"RelDistribution" stands for "metadata") is just theory, but it will
look a lot like RelMdCollation.


https://github.com/apache/incubator-calcite/blob/master/core/src/main/java/org/apache/calcite/rel/metadata/RelMdCollation.java

On Wed, Feb 11, 2015 at 6:47 PM, Ashutosh Chauhan <hashutosh@apache.org>
wrote:

I think RelDistribution as a member of Exchange (as in Julian proposal)
makes more sense as compared to mine at field level.

On Wed, Feb 11, 2015 at 6:42 PM, Jacques Nadeau <jacques@apache.org>

wrote:


Does RelMdDistribution exist or as theory?

Definitely inclined to "Exchange" over "Shuffle" as more

generic/consistent

with Volcano.

Note, looking at Ashutosh I wonder a little about the need to have range
versus hash at the field information level.  I'm sure we can come up

with a

use case but it seems to overcomplicate.  We already have space

exploration

problems when trying to do distribution subsumption for joins with trait
propagation so I would be cautious about adding more dimensions to that
attribute.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message