hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jesus Camacho Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21857) Sort conditions in a filter predicate to accelerate query processing
Date Wed, 12 Jun 2019 23:40:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16862551#comment-16862551
] 

Jesus Camacho Rodriguez commented on HIVE-21857:
------------------------------------------------

I have created HIVE-21867 to tackle sorting of semijoin filters.

> Sort conditions in a filter predicate to accelerate query processing
> --------------------------------------------------------------------
>
>                 Key: HIVE-21857
>                 URL: https://issues.apache.org/jira/browse/HIVE-21857
>             Project: Hive
>          Issue Type: Bug
>          Components: CBO
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: HIVE-21857.01.patch, HIVE-21857.02.patch, HIVE-21857.03.patch
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Following approach similar to http://db.cs.berkeley.edu/jmh/miscpapers/sigmod93.pdf .
> To reorder predicates in AND conditions, we could rank each of elements in the clauses
in increasing order based on following formula:
> {code}
> rank = (selectivity - 1) / cost per tuple
> {code}
> Similarly, for OR conditions:
> {code}
> rank = (-selectivity) / cost per tuple
> {code}
> Selectivity can be computed with FilterSelectivityEstimator. For cost per tuple, we will
need to come up with some heuristic based on how expensive is the evaluation of the functions
contained in that predicate. Custom UDFs could be annotated.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message