spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ruslan Dautkhanov (JIRA)" <>
Subject [jira] [Commented] (SPARK-26764) [SPIP] Spark Relational Cache
Date Mon, 25 Feb 2019 16:20:00 GMT


Ruslan Dautkhanov commented on SPARK-26764:

That seems to be closely related to Hive materialized views - implemented in Hive 3.2

> [SPIP] Spark Relational Cache
> -----------------------------
>                 Key: SPARK-26764
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: Adrian Wang
>            Priority: Major
>         Attachments: Relational+Cache+SPIP.pdf
> In modern database systems, relational cache is a common technology to boost ad-hoc queries.
While Spark provides cache natively, Spark SQL should be able to utilize the relationship
between relations to boost all possible queries. In this SPIP, we will make Spark be able
to utilize all defined cached relations if possible, without explicit substitution in user
query, as well as keep some user defined cache available in different sessions. Materialized
views in many database systems provide similar function.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message