hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <>
Subject [jira] [Commented] (HIVE-21071) Improve getInputSummary
Date Tue, 05 Feb 2019 22:01:00 GMT


Hive QA commented on HIVE-21071:

Here are the results of testing the latest attachment:

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15730 tests executed
*Failed tests:*
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testComplexQuery (batchId=261)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=261)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes (batchId=264)

Test results:
Console output:
Test logs:

Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed

This message is automatically generated.

ATTACHMENT ID: 12957666 - PreCommit-HIVE-Build

> Improve getInputSummary
> -----------------------
>                 Key: HIVE-21071
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: HiveServer2
>    Affects Versions: 3.0.0, 4.0.0, 3.1.1
>            Reporter: BELUGA BEHR
>            Assignee: BELUGA BEHR
>            Priority: Major
>         Attachments: HIVE-21071.1.patch, HIVE-21071.2.patch, HIVE-21071.3.patch, HIVE-21071.4.patch,
HIVE-21071.5.patch, HIVE-21071.6.patch, HIVE-21071.7.patch
> There is a global lock in the {{getInptSummary}} code, so it is important that it be
fast.  The current implementation has quite a bit of overhead that can be re-engineered.
> For example, the current implementation keeps a map of File Path to ContentSummary object.
 This map is populated by several threads concurrently. The method then loops through the
map, in a single thread, at the end to add up all of the ContentSummary objects and ignores
the paths.  The code can be be re-engineered to not use a map, or a collection at all, to
store the results and instead just keep a running tally.  By keeping a tally, there is no
{{O\(n)}} operation at the end to perform the addition.
> There are other things can be improved.  The method returns an object which is never
used anywhere, so change method to void return type.

This message was sent by Atlassian JIRA

View raw message