hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21787) Metastore table cache enhancement
Date Wed, 12 Jun 2019 20:06:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862425#comment-16862425
] 

Hive QA commented on HIVE-21787:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12971526/HIVE-21833.8.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 16054 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_addpartition]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_alter_db_owner]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_alter_db_owner_default]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_alter_drop_ptn]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_alter_table_exchange_partition_fail2]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_alter_table_exchange_partition_fail]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_create_macro1]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_create_tbl] (batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_create_view]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_createview] (batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_ctas2] (batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_ctas] (batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_delete_nodeletepriv]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_desc_table_nosel]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_drop_db_cascade]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_drop_db_empty]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_droppartition]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_explain] (batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_insert_noinspriv]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_insert_noselectpriv]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_insertoverwrite_nodel]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_insertpart_noinspriv]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_msck] (batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_not_owner_alter_tab_rename]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_not_owner_alter_tab_serdeprop]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_not_owner_drop_tab2]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_not_owner_drop_tab]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_not_owner_drop_view]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_rolehierarchy_privs]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_select] (batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_select_view]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_show_columns]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_show_parts_nosel]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_truncate] (batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_truncate_2] (batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_update_noupdatepriv]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_create_no_grant]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_create_no_select_perm]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_drop_other]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_no_select_perm]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_rebuild_no_grant]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[materialized_view_authorization_rebuild_other]
(batchId=100)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[temp_table_authorize_create_tbl]
(batchId=100)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/17540/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17540/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17540/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 43 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12971526 - PreCommit-HIVE-Build

> Metastore table cache enhancement
> ---------------------------------
>
>                 Key: HIVE-21787
>                 URL: https://issues.apache.org/jira/browse/HIVE-21787
>             Project: Hive
>          Issue Type: New Feature
>          Components: Standalone Metastore
>            Reporter: Sam An
>            Assignee: Sam An
>            Priority: Major
>         Attachments: HIVE-21787.1.patch, HIVE-21787.2.patch, HIVE-21787.3.patch, HIVE-21787.4.patch,
HIVE-21787.5.patch, HIVE-21787.6.patch, HIVE-21787.7.patch, HIVE-21833.8.patch
>
>
> Metastore currently uses black/white list to specify patterns of tables to load into
the cache. Cache is loaded in one shot "prewarm", and updated by a background thread. This
is not a very efficient design. 
> In this feature, we try to enhance the cache for Tables with LRU to improve cache utilization.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message