drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha" <asi...@maprtech.com>
Subject Re: Review Request 20741: Cost model to include distribution cost and enable new plans
Date Thu, 01 May 2014 01:54:59 GMT


> On April 29, 2014, 5:12 a.m., Jacques Nadeau wrote:
> > Additional thought... it would be nice if we had one or two tests specifically focused
on costing.  For example, confirm that we pick merge join when the data is pre-ordered by
the join key, same for stream agg, etc.  Otherwise, looks good.

Yes, would be nice... I have done some manual testing but need to think about what exactly
we want to test.. is it plan shape or costs themselves or both.  I will try to work on that
in the background while I get the current code changes out.  


- Aman


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20741/#review41683
-----------------------------------------------------------


On April 26, 2014, 1:54 a.m., Aman Sinha wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20741/
> -----------------------------------------------------------
> 
> (Updated April 26, 2014, 1:54 a.m.)
> 
> 
> Review request for drill, Jacques Nadeau and Jinfeng Ni.
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> 1. Added DrillCostBase that includes distribution (network) cost.
> 2. Added cost formulas for computeSelfCost() for various Exchange operators, joins, aggregations
etc. 
> 3. Added Prels for HashJoin, HashAggregate and generate plans for those.  
> 4. Added exchange Prels: BroadcastExchangePrel and a new type called HashToMergeExchangePrel.

> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 7e3b63d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToMergeExchange.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableConfig.java
98892c0 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillFilterRelBase.java
955729b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillProjectRelBase.java
cf3d188 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScanRelBase.java
b370352 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScreenRelBase.java
51ed442 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillCostBase.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelOptCost.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelOptCostFactory.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java
1492a28 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTrait.java
b75fb40 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTraitDef.java
c2ebb7a 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/FilterPrel.java
0fc3abd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashAggPrel.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashAggPrule.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrel.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrule.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashToMergeExchangePrel.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashToRandomExchangePrel.java
e5c9661 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrel.java
978a531 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrule.java
8298e50 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java
e6e99c0 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrel.java a945129

>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SingleMergeExchangePrel.java
d9431cc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SortPrel.java 344be4e

>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrel.java
c2880da 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrule.java
a561a61 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/TopNPrel.java e981a45

>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/UnionExchangePrel.java
f89cbaa 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java
8892a8f 
>   exec/java-exec/src/test/java/org/apache/drill/TestTpchSingleMode.java 1ccb65c 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestHashJoin.java
ba067e2 
>   exec/java-exec/src/test/resources/join/hj_exchanges.json PRE-CREATION 
>   exec/java-exec/src/test/resources/queries/tpch/01.sql f33416f 
> 
> Diff: https://reviews.apache.org/r/20741/diff/
> 
> 
> Testing
> -------
> 
> Existing unit tests.  Some TPCH tests show memory leaks but that is not directly related
to the costing changes...however those leaks would have to be resolved before enabling the
new plans.  However, I wanted to get this review request out to get feedback.  
> 
> 
> Thanks,
> 
> Aman Sinha
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message