hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-16038) MM tables: fix (or disable) inferring buckets
Date Wed, 08 Mar 2017 03:18:37 GMT

     [ https://issues.apache.org/jira/browse/HIVE-16038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin resolved HIVE-16038.
-------------------------------------
       Resolution: Fixed
    Fix Version/s: hive-14535

Would be very easy to fix for a particular MM ID, but there's no guarantee that other MM IDs
would conform to the inferred buckets, so I added comments and warnings and let it continue
to fail (by discarding the inferred data, as it does already when the job doesn't produce
the requisite number of files for a partition, see _dyn_part test).
I suspect similar issues may affect ACID tables and any other nested directory cases (and
some overwrites?).

If somebody cares about this feature it should be easy to fix based on the comment added in
the patch.

> MM tables: fix (or disable) inferring buckets
> ---------------------------------------------
>
>                 Key: HIVE-16038
>                 URL: https://issues.apache.org/jira/browse/HIVE-16038
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>             Fix For: hive-14535
>
>
> The following tests on minimr produce diffs if all tables are changed to MM:
> {noformat}
> infer_bucket_sort_dyn_part
> infer_bucket_sort_num_buckets
> infer_bucket_sort_merge
> infer_bucket_sort_reducers_power_two
> {noformat}
> Some of these disable strict checks for bucketing load, which wouldn't work by design;
the rest should work. Either that, or we should disable this for MM tables - seems like an
obscure feature.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message