hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20664) Potential ArrayIndexOutOfBoundsException in VectorizedOrcAcidRowBatchReader.findMinMaxKeys
Date Tue, 02 Oct 2018 14:43:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16635593#comment-16635593
] 

Hive QA commented on HIVE-20664:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12942089/HIVE-20664.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15001 tests executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=194)
	[druidmini_dynamic_partition.q,druidmini_test_ts.q,druidmini_expressions.q,druidmini_test_alter.q,druidmini_test_insert.q]
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14178/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14178/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14178/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12942089 - PreCommit-HIVE-Build

> Potential ArrayIndexOutOfBoundsException in VectorizedOrcAcidRowBatchReader.findMinMaxKeys
> ------------------------------------------------------------------------------------------
>
>                 Key: HIVE-20664
>                 URL: https://issues.apache.org/jira/browse/HIVE-20664
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>            Reporter: Saurabh Seth
>            Assignee: Saurabh Seth
>            Priority: Minor
>         Attachments: HIVE-20664.2.patch, HIVE-20664.patch
>
>
> [~ekoifman], could you please confirm if my understanding is correct and if so, review
the fix?
> In the method {{VectorizedOrcAcidRowBatchReader.findMinMaxKeys}}, the code snippet that
identifies the first and last stripe indices in the current split could result in an ArrayIndexOutOfBoundsException
if a complete split is within the same stripe:
> {noformat}
>     for(int i = 0; i < stripes.size(); i++) {
>       StripeInformation stripe = stripes.get(i);
>       long stripeEnd = stripe.getOffset() + stripe.getLength();
>       if(firstStripeIndex == -1 && stripe.getOffset() >= splitStart) {
>         firstStripeIndex = i;
>       }
>       if(lastStripeIndex == -1 && splitEnd <= stripeEnd &&
>           stripes.get(firstStripeIndex).getOffset() <= stripe.getOffset() ) {
>         //the last condition is for when both splitStart and splitEnd are in
>         // the same stripe
>         lastStripeIndex = i;
>       }
>     }
> {noformat}
> Consider the example where there are 2 stripes - 0-500 and 500-1000 and splitStart is
600 and splitEnd is 800.
> In the first iteration of the loop, stripe.getOffset() is 0 and stripeEnd is 500. In
this iteration, neither of the if statement conditions will be met and firstSripeIndex as
well as lastStripeIndex remain -1.
> In the second iteration of the loop stripe.getOffset() is 500, stripeEnd is 1000, The
first if statement condition will not be met in this case because stripe's offset (500) is
not greater than or equal to the splitStart (600). However, in the second if statement, splitEnd
(800) is <= stripeEnd(1000) and it will try to compute the last condition {{stripes.get(firstStripeIndex).getOffset()
<= stripe.getOffset()}}. This will throw an ArrayIndexOutOfBoundsException because firstStripeIndex
is still -1.
> I'm not sure if this scenario is possible at all, hence logging this as a low priority
issue. Perhaps block based split generation using BISplitStrategy could trigger this?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message