hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20661) Dynamic partitions loading calls add partition for every partition 1-by-1
Date Tue, 02 Oct 2018 18:14:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16635921#comment-16635921
] 

Hive QA commented on HIVE-20661:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12942087/HIVE-20661.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 55 failed/errored test(s), 15006 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[buckets] (batchId=268)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=268)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_buckets] (batchId=268)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[parquet_buckets] (batchId=268)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[rcfile_buckets] (batchId=268)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_insert_overwrite] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats2] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_3] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynamic_partition_insert] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lock3] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lock4] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_all] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_partitioned] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2] (batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part2] (batchId=21)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all] (batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_dp] (batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets] (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_mm] (batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_non_mm] (batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_opt_vectorization]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_merge_incompat2] (batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_input_counters] (batchId=179)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=191)
org.apache.hadoop.hive.ql.TestTxnExIm.testExportPart (batchId=298)
org.apache.hadoop.hive.ql.TestTxnExIm.testExportPartPartial (batchId=298)
org.apache.hadoop.hive.ql.TestTxnExIm.testExportPartPartial2 (batchId=298)
org.apache.hadoop.hive.ql.TestTxnExIm.testExportPartPartial3 (batchId=298)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportPartitionedCreate (batchId=298)
org.apache.hadoop.hive.ql.TestTxnExIm.testImportPartitionedCreate2 (batchId=298)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testNoBucketsDP (batchId=300)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testNoBucketsDP (batchId=300)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking7 (batchId=315)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidBootstrapReplLoadRetryAfterFailure
(batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrap
(batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesBootstrapWithOpenTxnsTimeout
(batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesMoveOptimizationBootStrap
(batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testAcidTablesMoveOptimizationIncremental
(batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testDumpAcidTableWithPartitionDirMissing
(batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testMultiDBTxn (batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testOpenTxnEvent (batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTables.testTxnEventNonAcid (batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosIncrementalLoadAcidTables.testAcidTableIncrementalReplication
(batchId=244)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosIncrementalLoadAcidTables.testReplCM
(batchId=244)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.schemaEvolutionAddColDynamicPartitioningInsert[0]
(batchId=241)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.schemaEvolutionAddColDynamicPartitioningInsert[1]
(batchId=241)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.schemaEvolutionAddColDynamicPartitioningUpdate[0]
(batchId=241)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.schemaEvolutionAddColDynamicPartitioningUpdate[1]
(batchId=241)
org.apache.hadoop.hive.ql.util.TestUpgradeTool.testPostUpgrade (batchId=276)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlInsertPartition (batchId=259)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=252)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14181/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14181/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14181/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 55 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12942087 - PreCommit-HIVE-Build

> Dynamic partitions loading calls add partition for every partition 1-by-1
> -------------------------------------------------------------------------
>
>                 Key: HIVE-20661
>                 URL: https://issues.apache.org/jira/browse/HIVE-20661
>             Project: Hive
>          Issue Type: Improvement
>          Components: Hive
>    Affects Versions: 4.0.0
>            Reporter: Laszlo Pinter
>            Assignee: Laszlo Pinter
>            Priority: Major
>         Attachments: HIVE-20661.01.patch, HIVE-20661.02.patch
>
>
> Hive.loadDynamicPartitions creates partitions using a threadpool, but the update of the
Metastore  via the MetastoreClient is done 1-by-1. This adds unnecessary extra calls. The
partitions should be created in one batch. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message