drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles Givre (Jira)" <j...@apache.org>
Subject [jira] [Created] (DRILL-7763) Add Limit Pushdown to File Based Storage Plugins
Date Fri, 03 Jul 2020 04:16:00 GMT
Charles Givre created DRILL-7763:
------------------------------------

             Summary: Add Limit Pushdown to File Based Storage Plugins
                 Key: DRILL-7763
                 URL: https://issues.apache.org/jira/browse/DRILL-7763
             Project: Apache Drill
          Issue Type: Improvement
    Affects Versions: 1.17.0
            Reporter: Charles Givre
            Assignee: Charles Givre
             Fix For: 1.18.0


As currently implemented, when querying a file, Drill will read the entire file even if a
limit is specified in the query.  This PR does a few things:
 # Refactors the EasyGroupScan, EasySubScan, and EasyFormatConfig to allow the option of pushing
down limits.
 # Applies this to all the EVF based format plugins which are: LogRegex, PCAP, SPSS, Esri,
Excel and Text (CSV). 

Due to JSON's fluid schema, it would be unwise to adopt the limit pushdown as it could result
in very inconsistent schemata.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message