hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17684) HoS memory issues with MapJoinMemoryExhaustionHandler
Date Wed, 20 Dec 2017 00:46:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297718#comment-16297718
] 

Hive QA commented on HIVE-17684:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12902915/HIVE-17684.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 267 failed/errored test(s), 10832 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.org.apache.hadoop.hive.cli.TestBeeLineDriver
(batchId=246)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin] (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_1] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_2] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_9] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join0] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join10] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join11] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join12] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join13] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join14] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join15] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join17] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join19_inclause] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join1] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join20] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join21] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join22] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join23] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join24] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join26] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join27] (batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join29] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join31] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join33] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join3] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join4] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join5] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join8] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join9] (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats] (batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_without_localtask] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_10] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_11] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_12] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_14] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_1] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_3] (batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_4] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_5] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_7] (batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_1] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_2] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark1] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark3] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark4] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_1] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_2] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_3] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_4] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_5] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_6] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_7] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_8] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin10] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin12] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin13] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin5] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin8] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin9] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_auto_join17] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer5] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer7] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_join2] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[identity_project_remove_skip] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_convert_join] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join25] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join26] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join27] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join28] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join29] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join30] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join31] (batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join32] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join33] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join34] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join35] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join36] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join37] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join39] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join40] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_unqual5] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_empty] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_map_ppr] (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_on_varchar] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_star] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin1] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin2] (batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin3] (batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin46] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin47] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_addjar] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_distinct] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_filter_on_outerjoin] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_mapjoin] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_subquery2] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_subquery] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_test_outer] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonblock_op_deduplicate] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_join] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[runtime_skewjoin_mapjoin_spark] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[select_transform_hint] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin10] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin11] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin1] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin2] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin3] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin4] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin5] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin6] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin7] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin8] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin9] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_46] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_47] (batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_4] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_6] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_7] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sort_merge_join_desc_8] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_multiinsert] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[tez_join_hash] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union22] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union34] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_12] (batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_13] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_14] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_between_columns] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_binary_join_groupby] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_char_mapjoin1] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_coalesce_3] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_mapjoin] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_mapjoin] (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_include_no_sel] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_interval_mapjoin] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_left_outer_join] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join0] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join1] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join2] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join3] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join4] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join6] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_reduce_groupby_duplicate_cols]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_varchar_mapjoin1] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_context] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_mapjoin2] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_mapjoin] (batchId=72)
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_ppd_join] (batchId=98)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketsortoptimize_insert_2]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] (batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_recursive_mapjoin]
(batchId=178)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_part] (batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1] (batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=104)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=105)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=106)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=109)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=112)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=116)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=118)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=123)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=124)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=126)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=129)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=132)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=134)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=135)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=136)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=137)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=138)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=139)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=140)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=141)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=142)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=143)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=144)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=145)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=146)
org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver
(batchId=248)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39] (batchId=248)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut (batchId=209)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testAutoPurgeInvalid (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testAutoPurgeUnset (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testExternalNoAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testExternalPartitionedTable (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testExternalTable (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testNoAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testPartitionedExternalNoAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testPartitionedNoAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testPartitionedTable (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateInvalidAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateNoAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncatePartitionedAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncatePartitionedNoAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testTruncateUnsetAutoPurge (batchId=228)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testMoreComplexQueryWithDirectSql
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testMoreComplexQueryWithDirectSqlTooManyPartitions
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithFallbackToORM1 (batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithFallbackToORM2 (batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithFallbackToORM3 (batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithFallbackToORMTooManyPartitions1
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithFallbackToORMTooManyPartitions2
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithFallbackToORMTooManyPartitions3
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithFallbackToORMTooManyPartitions4
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithInWithFallbackToORM
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithInWithFallbackToORMTooManyPartitions
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithInWithFallbackToORMTooManyPartitions2
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testQueryWithLikeWithFallbackToORMTooManyPartitions
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testSimpleQueryWithDirectSql
(batchId=226)
org.apache.hadoop.hive.ql.TestMetaStoreLimitPartitionRequest.testSimpleQueryWithDirectSqlTooManyPartitions
(batchId=226)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=226)
org.apache.hive.jdbc.TestJdbcWithLocalClusterSpark.testSparkQuery (batchId=236)
org.apache.hive.jdbc.TestJdbcWithLocalClusterSpark.testTempTable (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConcurrentStatements (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testEmptyResultsetThriftSerializeInTasks (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testEnableThriftSerializeInTasks (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testFloatCast2DoubleThriftSerializeInTasks (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testJoinThriftSerializeInTasks (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation2 (batchId=236)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testSelectThriftSerializeInTasks (batchId=236)
org.apache.hive.jdbc.TestMultiSessionsHS2WithLocalClusterSpark.testSparkQuery (batchId=236)
org.apache.hive.minikdc.TestHiveAuthFactory.org.apache.hive.minikdc.TestHiveAuthFactory (batchId=247)
org.apache.hive.minikdc.TestHs2HooksWithMiniKdc.org.apache.hive.minikdc.TestHs2HooksWithMiniKdc
(batchId=247)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc
(batchId=247)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.org.apache.hive.minikdc.TestJdbcWithDBTokenStore
(batchId=247)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs
(batchId=247)
org.apache.hive.minikdc.TestJdbcWithMiniKdc.org.apache.hive.minikdc.TestJdbcWithMiniKdc (batchId=247)
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie
(batchId=247)
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthBinary.org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthBinary
(batchId=247)
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp.org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp
(batchId=247)
org.apache.hive.minikdc.TestMiniHiveKdc.testLogin (batchId=247)
org.apache.hive.minikdc.TestSSLWithMiniKdc.org.apache.hive.minikdc.TestSSLWithMiniKdc (batchId=247)
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementParallel
(batchId=230)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8328/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8328/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8328/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 267 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12902915 - PreCommit-HIVE-Build

> HoS memory issues with MapJoinMemoryExhaustionHandler
> -----------------------------------------------------
>
>                 Key: HIVE-17684
>                 URL: https://issues.apache.org/jira/browse/HIVE-17684
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>            Reporter: Sahil Takiar
>            Assignee: Misha Dmitriev
>         Attachments: HIVE-17684.01.patch, HIVE-17684.02.patch
>
>
> We have seen a number of memory issues due the {{HashSinkOperator}} use of the {{MapJoinMemoryExhaustionHandler}}.
This handler is meant to detect scenarios where the small table is taking too much space in
memory, in which case a {{MapJoinMemoryExhaustionError}} is thrown.
> The configs to control this logic are:
> {{hive.mapjoin.localtask.max.memory.usage}} (default 0.90)
> {{hive.mapjoin.followby.gby.localtask.max.memory.usage}} (default 0.55)
> The handler works by using the {{MemoryMXBean}} and uses the following logic to estimate
how much memory the {{HashMap}} is consuming: {{MemoryMXBean#getHeapMemoryUsage().getUsed()
/ MemoryMXBean#getHeapMemoryUsage().getMax()}}
> The issue is that {{MemoryMXBean#getHeapMemoryUsage().getUsed()}} can be inaccurate.
The value returned by this method returns all reachable and unreachable memory on the heap,
so there may be a bunch of garbage data, and the JVM just hasn't taken the time to reclaim
it all. This can lead to intermittent failures of this check even though a simple GC would
have reclaimed enough space for the process to continue working.
> We should re-think the usage of {{MapJoinMemoryExhaustionHandler}} for HoS. In Hive-on-MR
this probably made sense to use because every Hive task was run in a dedicated container,
so a Hive Task could assume it created most of the data on the heap. However, in Hive-on-Spark
there can be multiple Hive Tasks running in a single executor, each doing different things.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message