drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-1928) Refactor filter pushdown code
Date Mon, 05 Jan 2015 18:24:34 GMT
Venki Korukanti created DRILL-1928:

             Summary: Refactor filter pushdown code
                 Key: DRILL-1928
                 URL: https://issues.apache.org/jira/browse/DRILL-1928
             Project: Apache Drill
          Issue Type: Improvement
          Components: Query Planning & Optimization
    Affects Versions: 0.7.0, 0.8.0
            Reporter: Venki Korukanti
            Assignee: Venki Korukanti
             Fix For: Future

Currently in many places (InfoSchema, HBase, MongoDB and FS/Hive partition pruning) we have
logic to push the filter into scan.

1. We can have common code for visiting the expression tree and determining which part of
the tree can be pushed to scan and which part can't be pushed to scan. 
2. In all places, if we can't convert the complete filter, partially converted filter is pushed
into Scan but the complete filter is copied and evaluated in Filter operator. This causes
the partially pushed filter to be evaluated in two places (Scan and Filter). 

This JIRA proposes following API:
 * @param filter Filter expression tree
 * @param fields List of columns names to consider when extracting the filter. For example
partition pruning is interested in only on expressions that involve partition columns. In
that case partition pruning can pass list of partition columns as supported column list.
 * @param functions List of supported functions to consider in extracting the filter. For
example partition pruning is interested in only "=", ">" etc. It can pass these functions
as the supported function list.
 * @return Result contains two trees. One tree that can be pushed to Scan and other tree that
can't be pushed into Scan. Either of them can be null.
FilterPruningResult extract(FilterExpression, FieldList, FunctionList)

This message was sent by Atlassian JIRA

View raw message