phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maryann Xue (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-2198) Support correlate variable
Date Fri, 04 Sep 2015 00:47:45 GMT


Maryann Xue commented on PHOENIX-2198:

In terms of performance, the number of iterations is fixed, but we could probably combine
the network requests or even HBase scans together for several iterations of the inner relation,
but this would be a really case-by-case optimization depending on what type of QueryPlan the
inner relation is. Basically it can only be a simple ScanPlan, and if it turns out that the
dynamic filter compiles into an HBase Get, we can batch these Get into one network request.
But if the inner plan is a range scan or a full scan with a filter, it becomes much trickier,
we might have to combine the filter (requires some compile-time work) with OR logic and push
them into one scan (otherwise it wouldn't make much sense), and then after the scan returns,
we'll have to figure out on the client side which results maps to which correlate variable
value, which is like executing the inner query again over a much smaller result set.
Anyway, I think we'll first just implement the literal semantics and make sure things all
work well with this basic version. And later on, we can think of optimizing CorrelatePlan
bit by bit.

Yes, VIEWs should just work fine. In Calcite VIEWs are replaced with its definition algebra
in the real query, so the filter of the VIEW will be automatically added to the relational
expression tree, and all those rules dealing with Filter (filter-combine-rule, filter-push-down-to-scan-rule,
etc) will kick in to make this work. But good reminder, James! I'll add a test on the calcite
branch to verify.
Global index hint doesn't work in Phoenix/Calcite integration yet. It depends on CALCITE-772.
But on the Phoenix side, it's a more general question of migrating our child/parent join optimization
onto calcite branch, which is logged as PHOENIX-1785. And this dynamic filter added in BaseQueryPlan
is just a good facility that can be made use of later on for this issue.

bq. Sometimes this can happen if the size of the row changes, as you'll send up with more
guideposts than before.
Can you explain a little more? How will the size of the row change in this test case? BTW,
I can't repro any more now.

> Support correlate variable
> --------------------------
>                 Key: PHOENIX-2198
>                 URL:
>             Project: Phoenix
>          Issue Type: New Feature
>            Reporter: Maryann Xue
>            Assignee: Maryann Xue
>         Attachments: PHOENIX-2198.patch
>   Original Estimate: 240h
>  Remaining Estimate: 240h
> This will enable the outer query to set a correlate variable as a parameter to restart
the inner query for each iteration.

This message was sent by Atlassian JIRA

View raw message