drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles Givre (Jira)" <j...@apache.org>
Subject [jira] [Created] (DRILL-7763) Add Limit Pushdown to File Based Storage Plugins
Date Fri, 03 Jul 2020 04:16:00 GMT
Charles Givre created DRILL-7763:

             Summary: Add Limit Pushdown to File Based Storage Plugins
                 Key: DRILL-7763
                 URL: https://issues.apache.org/jira/browse/DRILL-7763
             Project: Apache Drill
          Issue Type: Improvement
    Affects Versions: 1.17.0
            Reporter: Charles Givre
            Assignee: Charles Givre
             Fix For: 1.18.0

As currently implemented, when querying a file, Drill will read the entire file even if a
limit is specified in the query.  This PR does a few things:
 # Refactors the EasyGroupScan, EasySubScan, and EasyFormatConfig to allow the option of pushing
down limits.
 # Applies this to all the EVF based format plugins which are: LogRegex, PCAP, SPSS, Esri,
Excel and Text (CSV). 

Due to JSON's fluid schema, it would be unwise to adopt the limit pushdown as it could result
in very inconsistent schemata.

This message was sent by Atlassian Jira

View raw message