spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ioana Delaney (JIRA)" <>
Subject [jira] [Commented] (SPARK-16026) Cost-based Optimizer framework
Date Mon, 13 Feb 2017 23:50:43 GMT


Ioana Delaney commented on SPARK-16026:

[~Tagar], Our team is currently working on the support for informational RI constraints. We
plan to post the design soon, hopefully in the next couple of weeks. The design will cover
the constraint specification, metastore storage, constraint validation and maintenance. We
will also show examples of query optimizations that utilize RI constraints from simple join
and sort elimination to more complex Group By push down through join transformations, and
Star Schema detection.

> Cost-based Optimizer framework
> ------------------------------
>                 Key: SPARK-16026
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>            Reporter: Reynold Xin
>              Labels: releasenotes
>         Attachments: Spark_CBO_Design_Spec.pdf
> This is an umbrella ticket to implement a cost-based optimizer framework beyond broadcast
join selection. This framework can be used to implement some useful optimizations such as
join reordering.
> The design should discuss how to break the work down into multiple, smaller logical units.
For example, changes to statistics class, system catalog, cost estimation/propagation in expressions,
cost estimation/propagation in operators can be done in decoupled pull requests.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message