hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-18149) Stats: rownum estimation from datasize underestimates in most cases
Date Tue, 05 Dec 2017 13:13:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16278516#comment-16278516
] 

Hive QA commented on HIVE-18149:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12899660/HIVE-18149.01wip01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 333 failed/errored test(s), 11509 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[select_dummy_source] (batchId=247)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_1] (batchId=247)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_2] (batchId=247)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_3] (batchId=247)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[explain] (batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table] (batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table] (batchId=250)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5] (batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_5a] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats2] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join_stats] (batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_12] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_5] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[binarysortable_1] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_1] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_2] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_5] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative3] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[case_sensitivity] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join1] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_udaf_percentile_approx_23] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnarserde_create_shortcut] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_tbllvl] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[combine2] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[compute_stats_date] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[concat_op] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[correlationoptimizer5] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_precision2] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_precision] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_udf2] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_udf] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[display_colstats_tbllvl] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_windowing] (batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_windowing_no_cbo] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[drop_table_with_index] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_cond_pushdown2] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[gen_udf_example_add10] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby10] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_cube1] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_id3] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets1] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets2] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets3] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets4] (batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets5] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets6] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets_grouping] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_grouping_sets_limit] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_rollup1] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_11] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_serde] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_skewtable] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_const_type] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input17] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input21] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input22] (batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input3_limit] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input4] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input5] (batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_columnarserde] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_dynamicserde] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_lazyserde2] (batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_lazyserde] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath2] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath3] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath4] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_testxpath] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into1] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into2] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_hive_626] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_reorder2] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_reorder3] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_reorder4] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_reorder] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_star] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_thrift] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_subquery2] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullformatCTAS] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullscript] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge5] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge6] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat1] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_boolexpr] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ptf_matchpath] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_skew] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[regex_col] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[runtime_skewjoin_mapjoin_spark] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[select_dummy_source] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin11] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin1] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin2] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin3] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin4] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin5] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin6] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin7] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin8] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin9] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_union_remove_1] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_union_remove_2] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt10] (batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt11] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt12] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt13] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt14] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt16] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt17] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt18] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt19] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt1] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt20] (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt21] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt3] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt4] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt5] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt6] (batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt7] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt8] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt9] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_1] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_25] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_2] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_3] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats10] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_noscan_2] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_ppr_all] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[symlink_text_input_format] (batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_display_colstats_tbllvl]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_literal] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udaf_percentile_approx_23] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_add_months] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_aes_decrypt] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_aes_encrypt] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitwise_shiftleft] (batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitwise_shiftright] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_bitwise_shiftrightunsigned] (batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_case_thrift] (batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_cbrt] (batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_character_length] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_coalesce] (batchId=82)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_crc32] (batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_current_database] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_date_format] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_decode] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_factorial] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_from_utc_timestamp] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_in_file] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_isnull_isnotnull] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_last_day] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_length] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_levenshtein] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_first_n] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_hash] (batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_last_n] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_show_first_n] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_show_last_n] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_md5] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_months_between] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_nullif] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_octet_length] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_quarter] (batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_sha1] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_sha2] (batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_size] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_soundex] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_substring_index] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_to_utc_timestamp] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_trunc] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_trunc_number] (batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_width_bucket] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udtf_stack] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union21] (batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_10] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_11] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_12] (batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_13] (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_14] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_15] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_16] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_17] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_18] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_19] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_1] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_20] (batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_21] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_22] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_23] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_24] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_25] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_2] (batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_3] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_4] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_5] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_6] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_6_subq] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_7] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_8] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_remove_9] (batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_view] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_bucket] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_10_0] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_precision] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_udf2] (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_tablesample_rows] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_parquet_projection] (batchId=43)
org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[udf_example_arraymapstruct]
(batchId=242)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_partition_pruning_2]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_semijoin_user_level]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[insert_into1] (batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[insert_into2] (batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_partition_pruning]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_opt_vectorization]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_partitioned] (batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[partition_multilevels]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_complex_all] (batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_partitioned_date_time]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_partition_pruning]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[gen_udf_example_add10]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge5] (batchId=178)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge6] (batchId=178)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge7] (batchId=180)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_incompat1]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_incompat2]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[parallel_orderby] (batchId=178)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_2]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_3]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=177)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_map_operators]
(batchId=90)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[parallel_orderby] (batchId=90)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join_stats2] (batchId=144)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join_stats] (batchId=126)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_sortmerge_join_5] (batchId=144)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket_map_join_1] (batchId=134)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucket_map_join_2] (batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin4] (batchId=104)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin_negative3] (batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[column_access_stats] (batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[count] (batchId=116)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby10] (batchId=132)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_cube1] (batchId=105)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_rollup1] (batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[input17] (batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[insert_into1] (batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[insert_into2] (batchId=145)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_hive_626] (batchId=142)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_nullsafe] (batchId=137)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_reorder2] (batchId=112)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_reorder3] (batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_reorder4] (batchId=106)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_reorder] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_star] (batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[join_thrift] (batchId=138)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[mapjoin_subquery2] (batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ptf_matchpath] (batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[runtime_skewjoin_mapjoin_spark]
(batchId=129)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoin] (batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoin_union_remove_1] (batchId=144)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoin_union_remove_2] (batchId=117)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt10] (batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt11] (batchId=137)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt12] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt13] (batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt14] (batchId=137)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt16] (batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt17] (batchId=143)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt18] (batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt19] (batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt1] (batchId=141)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt20] (batchId=139)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt2] (batchId=106)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt3] (batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt4] (batchId=116)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt5] (batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt6] (batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt7] (batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt8] (batchId=116)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt9] (batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_1] (batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_25] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_2] (batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_3] (batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_4] (batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[smb_mapjoin_5] (batchId=145)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats10] (batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats_noscan_2] (batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] (batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union21] (batchId=104)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_10] (batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_11] (batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_12] (batchId=124)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_13] (batchId=144)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_14] (batchId=109)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_15] (batchId=143)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_16] (batchId=138)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_17] (batchId=135)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_18] (batchId=107)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_19] (batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_1] (batchId=119)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_20] (batchId=143)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_21] (batchId=132)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_22] (batchId=140)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_23] (batchId=138)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_24] (batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_25] (batchId=145)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_2] (batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_3] (batchId=129)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_4] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_5] (batchId=132)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_6] (batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_6_subq] (batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_7] (batchId=105)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_8] (batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_remove_9] (batchId=137)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_view] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_parquet_projection]
(batchId=124)
org.apache.hadoop.hive.ql.TestAcidOnTez.testMapJoinOnTez (batchId=224)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=227)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8109/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8109/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8109/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 333 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12899660 - PreCommit-HIVE-Build

> Stats: rownum estimation from datasize underestimates in most cases
> -------------------------------------------------------------------
>
>                 Key: HIVE-18149
>                 URL: https://issues.apache.org/jira/browse/HIVE-18149
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Statistics
>            Reporter: Zoltan Haindrich
>            Assignee: Zoltan Haindrich
>         Attachments: HIVE-18149.01wip01.patch
>
>
> rownum estimation is based on the following fact as of now:
> * datasize being used from the following sources:
> ** basicstats aggregates the loaded "on-heap" row sizes ; other readers are able to give
"raw size" estimation - I've checked orc; but I'm sure others will do the same....api docs
are a bit vague about the methods purpose...
> ** if the basicstats level info is not available; the filesystem level "file-size-sums"
are used as the "raw data size" ; which is multiplied by the [deserialization ratio|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L261]
; which is currently 1.
> the problem with all of this is that deser factor is 1; and that rowsize counts in the
online object headers..
> example; 20 rows are loaded into a partition [columnstats_partlvl_dp.q|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/test/queries/clientpositive/columnstats_partlvl_dp.q#L7]
> after HIVE-18108 [this explain|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/test/queries/clientpositive/columnstats_partlvl_dp.q#L25]
will estimate the rowsize of the table to be 404 bytes; however the 20 rows of text is only
169 bytes...so it ends up with 0 rows...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message