drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arina Ielchiieva (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5106) Refactor SkipRecordsInspector to exclude check for predefined file formats
Date Mon, 05 Dec 2016 12:18:58 GMT
Arina Ielchiieva created DRILL-5106:
---------------------------------------

             Summary: Refactor SkipRecordsInspector to exclude check for predefined file formats
                 Key: DRILL-5106
                 URL: https://issues.apache.org/jira/browse/DRILL-5106
             Project: Apache Drill
          Issue Type: Improvement
          Components: Storage - Hive
    Affects Versions: 1.9.0
            Reporter: Arina Ielchiieva
            Priority: Minor


After changes introduced in DRILL-4982, SkipRecordInspector is used only for predefined formats
(using hasHeaderFooter: false / true). But SkipRecordInspector has its own check for formats
where skip strategy can be applied. Acceptable file formats are stored in private final Set<Object>
fileFormats and initialized in constructor, currently it contains only one format - TextInputFormat.
Now this check is redundant and may lead to ignoring hasHeaderFooter setting true for any
other format except of Text.

To do:
1. remove private final Set<Object> fileFormats
2. remove if block from SkipRecordsInspector.retrievePositiveIntProperty:
{code}
 if (!fileFormats.contains(tableProperties.get(hive_metastoreConstants.FILE_INPUT_FORMAT)))
{
return propertyIntValue;
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message