drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha" <asi...@maprtech.com>
Subject Re: Review Request 20741: Cost model to include distribution cost and enable new plans
Date Thu, 01 May 2014 23:28:18 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20741/
-----------------------------------------------------------

(Updated May 1, 2014, 11:28 p.m.)


Review request for drill, Jacques Nadeau and Jinfeng Ni.


Changes
-------

Updated diffs that include the following changes: 
Addressed review comments. Added costing for OrderedPartitionExchange (this wasn't in my previous
changes). Removed obsolete method from Exchange interface.


Repository: drill-git


Description
-------

1. Added DrillCostBase that includes distribution (network) cost.
2. Added cost formulas for computeSelfCost() for various Exchange operators, joins, aggregations
etc. 
3. Added Prels for HashJoin, HashAggregate and generate plans for those.  
4. Added exchange Prels: BroadcastExchangePrel and a new type called HashToMergeExchangePrel.



Diffs (updated)
-----

  exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 7e3b63d 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/Exchange.java 80cf5a2 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/BroadcastExchange.java
256d3d9 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToMergeExchange.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToRandomExchange.java
61c2f1a 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/OrderedPartitionExchange.java
c49509f 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/SingleMergeExchange.java
df13a84 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/UnionExchange.java 78270d8

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillFilterRelBase.java
955729b 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillProjectRelBase.java
cf3d188 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScanRelBase.java
b370352 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScreenRelBase.java
51ed442 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillCostBase.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelOptCost.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelOptCostFactory.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillAggregateRel.java
c2833c1 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillAggregateRule.java
273237a 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java 1492a28

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/AggPruleBase.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTrait.java
b75fb40 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTraitDef.java
c2ebb7a 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/FilterPrel.java 0fc3abd

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashAggPrel.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashAggPrule.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrel.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrule.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashToMergeExchangePrel.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashToRandomExchangePrel.java
e5c9661 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/JoinPruleBase.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrel.java 978a531

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrule.java
8298e50 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/OrderedPartitionExchangePrel.java
b1dacc5 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java
e6e99c0 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrel.java a945129

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SingleMergeExchangePrel.java
d9431cc 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SortPrel.java 344be4e

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrel.java c2880da

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrule.java
a561a61 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/TopNPrel.java e981a45

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/UnionExchangePrel.java
f89cbaa 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java 8892a8f


Diff: https://reviews.apache.org/r/20741/diff/


Testing
-------

Existing unit tests.  Some TPCH tests show memory leaks but that is not directly related to
the costing changes...however those leaks would have to be resolved before enabling the new
plans.  However, I wanted to get this review request out to get feedback.  


Thanks,

Aman Sinha


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message