hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-18610) Performance: ListKeyWrapper does not check for hashcode equals, before comparing members
Date Thu, 01 Mar 2018 05:48:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381551#comment-16381551
] 

Gopal V edited comment on HIVE-18610 at 3/1/18 5:47 AM:
--------------------------------------------------------

This fixes the partial group-by hash function case where the ListKeyWrapper has a deeper Struct
(an Object[]) inside it.

This is not a correctness issue because if the partial group-by fails to de-dup, the reduce-side
GBY will treat identical rows from the same mapper as partial flushes & queries end up
producing the right result, but very slowly.


was (Author: gopalv):
This fixes the partial group-by hash function case where the ListKeyWrapper has a deeper Struct
inside it.

This is not a correctness issue because if the partial group-by fails to de-dup, the reduce-side
GBY will treat identical rows from the same mapper as partial flushes & queries end up
producing the right result, but very slowly.

> Performance: ListKeyWrapper does not check for hashcode equals, before comparing members
> ----------------------------------------------------------------------------------------
>
>                 Key: HIVE-18610
>                 URL: https://issues.apache.org/jira/browse/HIVE-18610
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>    Affects Versions: 3.0.0
>            Reporter: Gopal V
>            Assignee: Gopal V
>            Priority: Major
>         Attachments: HIVE-18610.1.patch, HIVE-18610.2.patch
>
>
> ListKeyWrapper::equals() 
> {code}
>     @Override
>     public boolean equals(Object obj) {
>       if (!(obj instanceof ListKeyWrapper)) {
>         return false;
>       }
>       Object[] copied_in_hashmap = ((ListKeyWrapper) obj).keys;
>       return equalComparer.areEqual(copied_in_hashmap, keys);
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message