hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17593) DataWritableWriter strip spaces for CHAR type before writing, but predicate generator doesn't do same thing.
Date Sat, 30 Jun 2018 03:00:02 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528513#comment-16528513
] 

ASF GitHub Bot commented on HIVE-17593:
---------------------------------------

GitHub user cjjnjust opened a pull request:

    https://github.com/apache/hive/pull/383

    HIVE-17593: DataWritableWriter strip spaces for CHAR type which causeā€¦

    Parquet DataWritableWriter strip tailing spaces for HiveChar type, which cause predicate
push down failed to work due to ConvertAstToSearchArg constructs predicate with tailing space.
 Actually, according to HiveChar definition, it should contains padded value. ParquetOutputFormat
can handle tailing spaces through encoding. 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/cjjnjust/hive HIVE-17593

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/hive/pull/383.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #383
    
----
commit 03230c732d657706c6a95f90e16ed5c81d411af7
Author: Chen, Junjie <cjjnjust@...>
Date:   2018-06-29T23:32:52Z

    HIVE-17593: DataWritableWriter strip spaces for CHAR type which cause PPD not work

----


> DataWritableWriter strip spaces for CHAR type before writing, but predicate generator
doesn't do same thing.
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-17593
>                 URL: https://issues.apache.org/jira/browse/HIVE-17593
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 2.3.0, 3.0.0
>            Reporter: Junjie Chen
>            Assignee: Junjie Chen
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.1.0
>
>         Attachments: HIVE-17593.patch
>
>
> DataWritableWriter strip spaces for CHAR type before writing. While when generating predicate,
it does NOT do same striping which should cause data missing!
> In current version, it doesn't cause data missing since predicate is not well push down
to parquet due to HIVE-17261.
> Please see ConvertAstTosearchArg.java, getTypes treats CHAR and STRING as same which
will build a predicate with tail spaces.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message