hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deepak Jaiswal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-19967) SMB Join : Need Optraits for PTFOperator ala GBY Op
Date Fri, 22 Jun 2018 05:27:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-19967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Deepak Jaiswal updated HIVE-19967:
----------------------------------
    Description: 
The SMB join on one or more PTF Ops should reset the optraits keys just like GBY Op does.

Currently there is no implementation of PTFOp optraits.

  was:
The optraits for ReduceSinkOp used to use the key columns as bucket and sort columns which
worked fine for SMB, however, to enable prefix in Bucket Map Join, this logic was updated
to use the bucket columns from parent operators. However, this may break reduce side SMB in
a scenario like this,

 

Task1 (TS bucketed by col0), passes it down to RS which ignores the key columns and uses col0
as bucket key.

Task2 (Set of ops work such that data is sorted by a set of columns), however, with current
logic, the bucketing column set in Task1 keeps getting pushed in Optraits, thus losing the
real flow.

Task3(Join op) The physical optimizer looks at the parent RS ops which incidentally are sorted
by same column as the original Task1's bucket column, however, in the meantime lost the meaning.

 

 

        Summary: SMB Join : Need Optraits for PTFOperator ala GBY Op  (was: SMB Join : ReduceSink
should use correct keys in optraits)

> SMB Join : Need Optraits for PTFOperator ala GBY Op
> ---------------------------------------------------
>
>                 Key: HIVE-19967
>                 URL: https://issues.apache.org/jira/browse/HIVE-19967
>             Project: Hive
>          Issue Type: Task
>            Reporter: Deepak Jaiswal
>            Assignee: Deepak Jaiswal
>            Priority: Major
>
> The SMB join on one or more PTF Ops should reset the optraits keys just like GBY Op does.
> Currently there is no implementation of PTFOp optraits.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message