hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vineet Garg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21991) Upgrade ORC version to 1.5.6
Date Mon, 22 Jul 2019 18:30:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16890380#comment-16890380
] 

Vineet Garg commented on HIVE-21991:
------------------------------------

Multiple tests in {{TestInputOutputFormat}} are failing due to less number of read operations
(specifically read ops are reduced by 2 now).

 {{testACIDReaderNoFooterSerialize}} is doing the following:
 * Create 2 files and write data to it.
 * Call getSplits on the dir. Number of read ops here are as before (3)
 * Call getRecordReader on each split and check the number of read ops.

This test expects number of read ops to be 8 for the last call (getRecordReader) for all splits
{noformat}
// call-1: open to read footer - split 1 => mock:/mocktable5/0_0
    // call-2: open to read data - split 1 => mock:/mocktable5/0_0
    // call-3: getAcidState - split 1 => mock:/mocktable5 (to compute offset for original
read)
    // call-4: open to read footer - split 2 => mock:/mocktable5/0_1
    // call-5: open to read data - split 2 => mock:/mocktable5/0_1
    // call-6: getAcidState - split 2 => mock:/mocktable5 (to compute offset for original
read)
    // call-7: open to read footer - split 2 => mock:/mocktable5/0_0 (to get row count)
    // call-8: file status - split 2 => mock:/mocktable5/0_0
{noformat}
But number of read ops are 6 instead of 8.

I don't understand call-7 and call-8 and 6 read ops make sense to me. So either this was a
bug before and is now fixed or number of read ops are wrong now.

[~prasanth_j] [~gopalv] [~bslim] Can you please help me understand this? How can I debug this
further to find out what exact read operations are being done? Are there trace logs which
I can turn on?

> Upgrade ORC version to 1.5.6
> ----------------------------
>
>                 Key: HIVE-21991
>                 URL: https://issues.apache.org/jira/browse/HIVE-21991
>             Project: Hive
>          Issue Type: Task
>          Components: ORC
>            Reporter: Vineet Garg
>            Assignee: Vineet Garg
>            Priority: Major
>         Attachments: HIVE-21991.1.patch, HIVE-21991.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message