drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arina Ielchiieva (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (DRILL-5106) Refactor SkipRecordsInspector to exclude check for predefined file formats
Date Thu, 23 Nov 2017 13:29:00 GMT

     [ https://issues.apache.org/jira/browse/DRILL-5106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arina Ielchiieva resolved DRILL-5106.
-------------------------------------
    Resolution: Fixed

Merged into Apache master with commit id f8bbb7591235627a58a1f10584d7902d8251d221

> Refactor SkipRecordsInspector to exclude check for predefined file formats
> --------------------------------------------------------------------------
>
>                 Key: DRILL-5106
>                 URL: https://issues.apache.org/jira/browse/DRILL-5106
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - Hive
>    Affects Versions: 1.9.0
>            Reporter: Arina Ielchiieva
>            Assignee: Arina Ielchiieva
>            Priority: Minor
>             Fix For: 1.12.0
>
>
> After changes introduced in DRILL-4982, SkipRecordInspector is used only for predefined
formats (using hasHeaderFooter: false / true). But SkipRecordInspector has its own check for
formats where skip strategy can be applied. Acceptable file formats are stored in private
final Set<Object> fileFormats and initialized in constructor, currently it contains
only one format - TextInputFormat. Now this check is redundant and may lead to ignoring hasHeaderFooter
setting to true for any other format except of Text.
> To do:
> 1. remove private final Set<Object> fileFormats
> 2. remove if block from SkipRecordsInspector.retrievePositiveIntProperty:
> {code}
>  if (!fileFormats.contains(tableProperties.get(hive_metastoreConstants.FILE_INPUT_FORMAT)))
{
> return propertyIntValue;
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message