hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19113) Bucketing: Make CLUSTERED BY do CLUSTER BY if no explicit sorting is specified
Date Fri, 19 Jul 2019 00:15:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888409#comment-16888409
] 

Hive QA commented on HIVE-19113:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12975197/HIVE-19113.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 63 failed/errored test(s), 16681 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_11] (batchId=295)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=298)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions]
(batchId=298)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid2] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[load_dyn_part2] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[load_static_ptn_into_bucketed_table]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck] (batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_buckets] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_11] (batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_21] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats10] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_bucket] (batchId=29)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning] (batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_no_buckets] (batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[acid_vectorization_original]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket2] (batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket3] (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_many] (batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_num_reducers2]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_num_reducers] (batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[check_constraint] (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint] (batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[disable_merge_for_bucketing]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_into_default_keyword]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[load_dyn_part2] (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[murmur_hash_migration]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sample10_mm] (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_hint] (batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge] (batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge_stats] (batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_fixed_bucket_pruning]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_bucket] (batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_insert_into_bucketed_table]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[disable_merge_for_bucketing]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[acid_vectorization_original_tez]
(batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket2] (batchId=135)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket3] (batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[disable_merge_for_bucketing] (batchId=148)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[dynpart_sort_optimization] (batchId=136)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[load_dyn_part2] (batchId=139)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_11] (batchId=112)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_21] (batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats10] (batchId=122)
org.apache.hadoop.hive.ql.TestAcidOnTez.testBucketedAcidInsertWithRemoveUnion (batchId=244)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCrudMajorCompactionSplitGrouper (batchId=244)
org.apache.hadoop.hive.ql.TestTxnAddPartition.addPartitionBucketed (batchId=345)
org.apache.hadoop.hive.ql.TestTxnAddPartition.addPartitionRename (batchId=345)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeOnTezEdges (batchId=349)
org.apache.hadoop.hive.ql.TestTxnCommands.testNonAcidToAcidConversion01 (batchId=349)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02 (batchId=330)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion02
(batchId=344)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
(batchId=331)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testNonAcidToAcidConversion01
(batchId=331)
org.apache.hadoop.hive.ql.TestTxnConcatenate.testConcatenate (batchId=316)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testToAcidConversionMultiBucket (batchId=331)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testToAcidConversionMultiBucket (batchId=331)
org.apache.hadoop.hive.ql.txn.compactor.TestCrudCompactorOnTez.testMajorCompaction (batchId=273)
org.apache.hadoop.hive.ql.util.TestUpgradeTool.testPostUpgrade (batchId=306)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/18096/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18096/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18096/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 63 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12975197 - PreCommit-HIVE-Build

> Bucketing: Make CLUSTERED BY do CLUSTER BY if no explicit sorting is specified
> ------------------------------------------------------------------------------
>
>                 Key: HIVE-19113
>                 URL: https://issues.apache.org/jira/browse/HIVE-19113
>             Project: Hive
>          Issue Type: Improvement
>          Components: Logical Optimizer
>    Affects Versions: 3.0.0
>            Reporter: Gopal V
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>         Attachments: HIVE-19113.patch
>
>
> The user's expectation of 
> "create external table bucketed (key int) clustered by (key) into 4 buckets stored as
orc;"
> is that the table will cluster the key into 4 buckets, while the file layout does not
do any actual clustering of rows.
> In the absence of a "SORTED BY", this can automatically do a "SORTED BY (key)" to cluster
the keys within the file as expected.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

Mime
View raw message