hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11807) Set ORC buffer size in relation to set stripe size
Date Sat, 19 Sep 2015 13:45:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14877125#comment-14877125
] 

Hive QA commented on HIVE-11807:
--------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12761202/HIVE-11807.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9449 tests executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
org.apache.hive.hcatalog.streaming.TestStreaming.testRemainingTransactions
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5341/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5341/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5341/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12761202 - PreCommit-HIVE-TRUNK-Build

> Set ORC buffer size in relation to set stripe size
> --------------------------------------------------
>
>                 Key: HIVE-11807
>                 URL: https://issues.apache.org/jira/browse/HIVE-11807
>             Project: Hive
>          Issue Type: Improvement
>          Components: File Formats
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>         Attachments: HIVE-11807.patch, HIVE-11807.patch
>
>
> A customer produced ORC files with very small stripe sizes (10k rows/stripe) by setting
a small 64MB stripe size and 256K buffer size for a 54 column table. At that size, each of
the streams only get a buffer or two before the stripe size is reached. The current code uses
the available memory instead of the stripe size and thus doesn't shrink the buffer size if
the JVM has much more memory than the stripe size.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message