hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7239) Fix bug in HiveIndexedInputFormat implementation that causes incorrect query result when input backed by Sequence/RC files
Date Thu, 28 Jul 2016 16:33:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15397774#comment-15397774
] 

Hive QA commented on HIVE-7239:
-------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12820366/HIVE-7239.4.patch

{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10391 tests executed
*Failed tests:*
{noformat}
TestMsgBusConnection - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_nullable_union
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_multiinsert
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_avro_non_nullable_union
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/674/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/674/console
Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-674/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12820366 - PreCommit-HIVE-MASTER-Build

> Fix bug in HiveIndexedInputFormat implementation that causes incorrect query result when
input backed by Sequence/RC files
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-7239
>                 URL: https://issues.apache.org/jira/browse/HIVE-7239
>             Project: Hive
>          Issue Type: Bug
>          Components: Indexing
>    Affects Versions: 2.1.0
>            Reporter: Sumit Kumar
>            Assignee: Illya Yalovyy
>         Attachments: HIVE-7239.2.patch, HIVE-7239.3.patch, HIVE-7239.4.patch, HIVE-7239.patch
>
>
> In case of sequence files, it's crucial that splits are calculated around the boundaries
enforced by the input sequence file. However by default hadoop creates input splits depending
on the configuration parameters which may not match the boundaries for the input sequence
file. Hive provides HiveIndexedInputFormat that provides extra logic and recalculates the
split boundaries for each split depending on the sequence file's boundaries.
> However we noticed this behavior of "over" reporting from data backed by sequence file.
We've a sample data on which we experimented and fixed this bug, we have verified this fix
by comparing the query output for input being sequence file format, rc file and regular format.
However we have not able to find the right place to include this as a unit test that would
execute as part of hive tests. We tried writing a "clientpositive" test as part of ql module
but the output seems quite verbose and i couldn't interpret it that well. Can someone please
review this change and guide on how to write a test that will execute as part of Hive testing?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message