hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Laljo John Pullokkaran (JIRA)" <>
Subject [jira] [Commented] (HIVE-12478) Improve Hive/Calcite Trasitive Predicate inference
Date Tue, 08 Dec 2015 23:00:11 GMT


Laljo John Pullokkaran commented on HIVE-12478:

Once HIVE-11918, HIVE-11110 goes in we should invoke PPD, JoinTransitive Inference & Constant
folding as one rule.
This rule would fire recursively until all the transitive inference is done.
One of the challenge is to recognize that we have pushed a filter already and not try to push
it again.

We have three options:
1. Use RelMDPredicates in calcite
2. Implement a  metadata provider that can answer if a predicate got already pushed (similar
to RelMDPredicates in calcite)
3. Enhance RuleRegistry to capture if an expression already pushed pat an operator.

#1 doesn't work when expressions might have transformed due to constant folding
#2 would be better; but it kind of goes against current Calcite design which assumes metadata
to be derivable (i.e no state maintenance)

That leaves us with #3.

[~jcamachorodriguez] Could you take a look

> Improve Hive/Calcite Trasitive Predicate inference
> --------------------------------------------------
>                 Key: HIVE-12478
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.2.1
>            Reporter: Laljo John Pullokkaran
>            Assignee: Laljo John Pullokkaran
> HiveJoinPushTransitivePredicatesRule does not pull up predicates for transitive inference
if they contain more than one column.
> EXPLAIN select * from srcpart join (select ds as ds, ds as `date` from srcpart where
 (ds = '2008-04-08' and value=1)) s on (srcpart.ds = s.ds);

This message was sent by Atlassian JIRA

View raw message