[ https://issues.apache.org/jira/browse/HIVE-10036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14491458#comment-14491458 ] Hive QA commented on HIVE-10036: -------------------------------- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12724721/HIVE-10036.5.patch {color:red}ERROR:{color} -1 due to 316 failed/errored test(s), 8672 tests executed *Failed tests:* {noformat} TestMinimrCliDriver-bucketmapjoin6.q-constprog_partitioner.q-infer_bucket_sort_dyn_part.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-external_table_with_space_in_location_path.q-infer_bucket_sort_merge.q-auto_sortmerge_join_16.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-groupby2.q-import_exported_table.q-bucketizedhiveinputformat.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-index_bitmap3.q-stats_counter_partitioned.q-temp_table_external.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_map_operators.q-join1.q-bucketmapjoin7.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_num_buckets.q-disable_merge_for_bucketing.q-uber_reduce.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-infer_bucket_sort_reducers_power_two.q-scriptfile1.q-scriptfile1_win.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-leftsemijoin_mr.q-load_hdfs_file_with_space_in_the_name.q-root_dir_external_table.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-list_bucket_dml_10.q-bucket_num_reducers.q-bucket6.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-load_fs2.q-file_with_header_footer.q-ql_rewrite_gbtoidx_cbo_1.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-parallel_orderby.q-reduce_deduplicate.q-ql_rewrite_gbtoidx_cbo_2.q-and-1-more - did not produce a TEST-*.xml file TestMinimrCliDriver-ql_rewrite_gbtoidx.q-smb_mapjoin_8.q - did not produce a TEST-*.xml file TestMinimrCliDriver-schemeAuthority2.q-bucket4.q-input16_cc.q-and-1-more - did not produce a TEST-*.xml file org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_join org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_partition org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_acid_vectorization_project org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_2_orc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_orc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_merge_stats_orc org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_filter org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_groupby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_limit org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_part org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_select org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_annotate_stats_union org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_delete org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_delete_own_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_update org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_update_own_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_char_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_date_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_decimal_join2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_all_non_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_all_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_orig_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_tmp_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_where_no_match org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_where_non_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_where_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_delete_whole_partition org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_opt_vectorization org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_optimization2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_optimization_acid org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_full org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_partial org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_extrapolate_part_stats_partial_ndv org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_implicit_cast_during_insert org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_acid_dynamic_partition org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_acid_not_bucketed org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_into_with_schema org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_nonacid_from_acid org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_orig_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_update_delete org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_values_acid_not_bucketed org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_values_dynamic_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_values_non_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_values_orig_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_values_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_values_tmp_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_analyze org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_create org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_dictionary_threshold org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_diff_part_cols org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_diff_part_cols2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_empty_files org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_empty_strings org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ends_with_nulls org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge7 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge8 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge_incompat1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_merge_incompat2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_min_max org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_null_check org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_boolean org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_char org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_decimal org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_timestamp org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_ppd_varchar org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_predicate_pushdown org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_vectorization_ppd org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_wide_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_temp_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_temp_table_options1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_acid org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_after_multiple_inserts org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_all_non_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_all_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_all_types org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_orig_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_tmp_table org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_two_cols org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_where_no_match org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_where_non_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_update_where_partitioned org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_varchar_serde org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_aggregate_9 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_between_in org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_cast_constant org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_char_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_char_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_char_mapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_char_simple org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_coalesce_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_count_distinct org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_data_types org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_date_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_10_0 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_6 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_aggregate org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_expressions org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_mapjoin org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_math_funcs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_precision org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_round org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_round_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_trailing org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_udf org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_decimal_udf2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_distinct_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_groupby_3 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_groupby_reduce org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_interval_1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_interval_2 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_multi_insert org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_non_string_partition org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_orderby_5 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_partition_diff_num_cols org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_partitioned_date_time org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_reduce_groupby_decimal org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_string_concat org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_varchar_4 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_varchar_mapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vector_varchar_simple org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_decimal_date org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_part org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_part_project org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_bucketmapjoin1 org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_context org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_date_funcs org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_distinct_gby org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_ptf org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_timestamp_funcs org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_static org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_2_orc org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_orc org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_alter_merge_stats_orc org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_all_non_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_all_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_orig_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_tmp_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_where_no_match org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_where_non_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_where_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_delete_whole_partition org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_opt_vectorization org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_explainuser_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_hybridhashjoin org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_orig_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_update_delete org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_values_dynamic_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_values_non_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert_values_tmp_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mapjoin_decimal org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_analyze org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge6 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge7 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge8 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge_incompat1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_merge_incompat2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_orc_vectorization_ppd org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_temp_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_bmj_schema_evolution org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_join_hash org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_schema_evolution org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_after_multiple_inserts org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_all_non_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_all_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_all_types org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_orig_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_tmp_table org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_two_cols org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_where_no_match org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_where_non_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_update_where_partitioned org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_between_in org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_cast_constant org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_char_4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_char_mapjoin1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_char_simple org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_coalesce_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_count_distinct org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_data_types org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_10_0 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_6 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_aggregate org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_expressions org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_mapjoin org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_math_funcs org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_precision org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_trailing org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_udf org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_decimal_udf2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_distinct_2 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_groupby_3 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_groupby_reduce org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_non_string_partition org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_orderby_5 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_partition_diff_num_cols org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_partitioned_date_time org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_reduce_groupby_decimal org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_string_concat org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_varchar_4 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_varchar_mapjoin1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_varchar_simple org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_decimal_date org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_part org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_part_project org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorization_short_regress org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_bucketmapjoin1 org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_context org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_date_funcs org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_distinct_gby org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_dynamic_partition_pruning org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_ptf org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vectorized_timestamp_funcs org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_acid_overwrite org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testMemoryManagementV12[0] org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testMemoryManagementV12[1] org.apache.hive.hcatalog.mapreduce.TestHCatDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask[3] org.apache.hive.hcatalog.mapreduce.TestHCatDynamicPartitioned.testHCatDynamicPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask[3] org.apache.hive.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatDynamicPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatExternalDynamicPartitioned.testHCatExternalDynamicCustomLocation[3] org.apache.hive.hcatalog.mapreduce.TestHCatExternalNonPartitioned.testHCatNonPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatExternalPartitioned.testHCatPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatMutableDynamicPartitioned.testHCatDynamicPartitionedTableMultipleTask[3] org.apache.hive.hcatalog.mapreduce.TestHCatMutableDynamicPartitioned.testHCatDynamicPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatMutableNonPartitioned.testHCatNonPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatMutablePartitioned.testHCatPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatNonPartitioned.testHCatNonPartitionedTable[3] org.apache.hive.hcatalog.mapreduce.TestHCatPartitioned.testHCatPartitionedTable[3] org.apache.hive.hcatalog.pig.TestE2EScenarios.testReadOrcAndRCFromPig org.apache.hive.hcatalog.pig.TestHCatLoader.testProjectionsBasic[3] org.apache.hive.hcatalog.pig.TestHCatLoader.testReadDataBasic[3] org.apache.hive.hcatalog.pig.TestHCatLoader.testReadPartitionedBasic[3] org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testMapNullKey[3] org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testMapWithComplexData[3] org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testSyntheticComplexSchema[3] org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testBagNStruct[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testDateCharTypes[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsInDataNoSpec[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testDynamicPartitioningMultiPartColsInDataPartialSpec[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testMultiPartColsInData[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testPartColsInData[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreFuncAllSimpleTypes[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreFuncSimple[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreInPartiitonedTbl[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreMultiTables[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreWithNoCtorArgs[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testStoreWithNoSchema[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteChar[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDate2[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDate3[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDate[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDecimalXY[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDecimalX[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteDecimal[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteSmallint[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteTimestamp[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteTinyint[3] org.apache.hive.hcatalog.pig.TestHCatStorer.testWriteVarchar[3] org.apache.hive.hcatalog.pig.TestHCatStorerMulti.testStoreBasicTable[3] org.apache.hive.hcatalog.pig.TestHCatStorerMulti.testStorePartitionedTable[3] org.apache.hive.hcatalog.pig.TestHCatStorerMulti.testStoreTableMulti[3] org.apache.hive.hcatalog.streaming.TestStreaming.testConcurrentTransactionBatchCommits org.apache.hive.hcatalog.streaming.TestStreaming.testInterleavedTransactionBatchCommits org.apache.hive.hcatalog.streaming.TestStreaming.testMultipleTransactionBatchCommits org.apache.hive.hcatalog.streaming.TestStreaming.testRemainingTransactions org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchAbort org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchAbortAndCommit org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Delimited org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Json org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyAbort org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit org.apache.hive.jdbc.TestJdbcWithMiniHS2.testNewConnectionConfiguration {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3394/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3394/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3394/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 316 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12724721 - PreCommit-HIVE-TRUNK-Build > Writing ORC format big table causes OOM - too many fixed sized stream buffers > ----------------------------------------------------------------------------- > > Key: HIVE-10036 > URL: https://issues.apache.org/jira/browse/HIVE-10036 > Project: Hive > Issue Type: Improvement > Reporter: Selina Zhang > Assignee: Selina Zhang > Attachments: HIVE-10036.1.patch, HIVE-10036.2.patch, HIVE-10036.3.patch, HIVE-10036.5.patch > > > ORC writer keeps multiple out steams for each column. Each output stream is allocated fixed size ByteBuffer (configurable, default to 256K). For a big table, the memory cost is unbearable. Specially when HCatalog dynamic partition involves, several hundreds files may be open and writing at the same time (same problems for FileSinkOperator). > Global ORC memory manager controls the buffer size, but it only got kicked in at 5000 rows interval. An enhancement could be done here, but the problem is reducing the buffer size introduces worse compression and more IOs in read path. Sacrificing the read performance is always not a good choice. > I changed the fixed size ByteBuffer to a dynamic growth buffer which up bound to the existing configurable buffer size. Most of the streams does not need large buffer so the performance got improved significantly. Comparing to Facebook's hive-dwrf, I monitored 2x performance gain with this fix. > Solving OOM for ORC completely maybe needs lots of effort , but this is definitely a low hanging fruit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)