[ https://issues.apache.org/jira/browse/HIVE-24221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17206086#comment-17206086 ] 

Stamatis Zampetakis commented on HIVE-24221:
--------------------------------------------

There are various ways to create a hash from composite keys/columns. Without any special effort to derive a perfect hash function we can do the following:

Input columns: A, B, C, D

+Option A:+ 
{noformat}
hash(hash(hash(A, B), C), D)
{noformat}

+Option B:+ 
{noformat}
31*(31*(31 * hash(A) + hash(B)) + hash(C)) + hash(D)
{noformat}

The second option is more or less what happens currently when we write hash(A, B, C, D) in the non-vectorized implementation of GenericUDFMurmurHash.
The first option although it looks simpler is computationally more expensive.

> Use vectorizable expression to combine multiple columns in semijoin bloom filters
> ---------------------------------------------------------------------------------
>
>                 Key: HIVE-24221
>                 URL: https://issues.apache.org/jira/browse/HIVE-24221
>             Project: Hive
>          Issue Type: Improvement
>          Components: Query Planning
>         Environment: 
>            Reporter: Stamatis Zampetakis
>            Assignee: Stamatis Zampetakis
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently, multi-column semijoin reducers use an n-ary call to GenericUDFMurmurHash to combine multiple values into one, which is used as an entry to the bloom filter. However, there are no vectorized operators that treat n-ary inputs. The same goes for the vectorized implementation of GenericUDFMurmurHash introduced in HIVE-23976. 
> The goal of this issue is to choose an alternative way to combine multiple values into one to pass in the bloom filter comprising only vectorized operators.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)