Charles Givre created DRILL-7763:
------------------------------------
Summary: Add Limit Pushdown to File Based Storage Plugins
Key: DRILL-7763
URL: https://issues.apache.org/jira/browse/DRILL-7763
Project: Apache Drill
Issue Type: Improvement
Affects Versions: 1.17.0
Reporter: Charles Givre
Assignee: Charles Givre
Fix For: 1.18.0
As currently implemented, when querying a file, Drill will read the entire file even if a
limit is specified in the query. This PR does a few things:
# Refactors the EasyGroupScan, EasySubScan, and EasyFormatConfig to allow the option of pushing
down limits.
# Applies this to all the EVF based format plugins which are: LogRegex, PCAP, SPSS, Esri,
Excel and Text (CSV).
Due to JSON's fluid schema, it would be unwise to adopt the limit pushdown as it could result
in very inconsistent schemata.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
|