drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinfeng Ni" <...@maprtech.com>
Subject Re: Review Request 20762: DRILL-491 : Push project into scan operator.
Date Thu, 01 May 2014 01:14:29 GMT


> On April 29, 2014, 5:25 p.m., Aditya Kishore wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushProjIntoScan.java,
line 63
> > <https://reviews.apache.org/r/20762/diff/1/?file=569306#file569306line63>
> >
> >     minor: Pleas avoid trailing white spaces in new code.

Thanks! Now I enable the option of removing trailing white space and unused imports in Eclipse.



> On April 29, 2014, 5:25 p.m., Aditya Kishore wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrule.java,
line 86
> > <https://reviews.apache.org/r/20762/diff/1/?file=569313#file569313line86>
> >
> >     How would it handle hierarchical column names? like 'family.column_a'?

This seems to be related to map, which is currently wip. I'll revisit this, if the design
of map is finalized. 


- Jinfeng


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20762/#review41743
-----------------------------------------------------------


On April 28, 2014, 10:12 a.m., Jinfeng Ni wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/20762/
> -----------------------------------------------------------
> 
> (Updated April 28, 2014, 10:12 a.m.)
> 
> 
> Review request for drill.
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> This patch is to push projected columns into group scan operator.
> 
> Tested with Parquet scan and JSON scan operator. 
> Currently, project pushdown is disabled for Text storage plugin, and Hive storage plugin.
> 
> Project pushdown happens at logical planning phase. Whether optimizer will pick choose
the plan with project-pushed down depends on the cost model and comparison.
> 
> 
> Diffs
> -----
> 
>   contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseStoragePlugin.java
a82c6c3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScanRelBase.java
b370352 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillProjectRel.java
d19b7a4 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushProjIntoScan.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java
1492a28 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillScanRel.java
619e76d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillScanPrel.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/GroupScanPrel.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PrelUtil.java c8412ab

>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrel.java a945129

>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrule.java
8985a58 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractStoragePlugin.java
3081b46 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePlugin.java 67f6df3

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
97427f6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FormatPlugin.java 73e414c

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/easy/EasyFormatPlugin.java
6e87da5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/TextFormatPlugin.java
850f248 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java
cff075c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveScan.java bf33805

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveStoragePlugin.java
e6df669 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveSubScan.java 8ff7c82

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/hive/HiveTextRecordReader.java
1e47684 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/ischema/InfoSchemaStoragePlugin.java
313ea86 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockStorageEngine.java
10f6e08 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java
d9e6795 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetGroupScan.java
cd7575d 
>   exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java a3a5647 
>   exec/java-exec/src/test/java/org/apache/drill/TestExampleQueries.java e0e874b 
>   exec/java-exec/src/test/java/org/apache/drill/TestProjectPushDown.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/20762/diff/
> 
> 
> Testing
> -------
> 
> Add one more unit test case : TestProjectPushDown to check the physical plan generated
from the optimizer.
> 
> When run mvn install against existing test suite, there are some testcases failures,
complaining running out time. However, those testcases will be successful if run within Eclipse
with assertion enabled ( -ea ).   
> 
> Need further figure out why these testcases fails when run through mvn command line.

> 
> Update:
> 
> run mvn install against all existing test suite successfully, after disabling "fork"
in root pom.xml. 
> 
> 
> Thanks,
> 
> Jinfeng Ni
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message