spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Zhuge <jzh...@apache.org>
Subject SPIP: Catalog API for view metadata
Date Wed, 22 Apr 2020 07:24:05 GMT
Hi everyone,

In order to disassociate view metadata from Hive Metastore and support
different storage backends, I am proposing a new view catalog API to load,
create, alter, and drop views.

Document:
https://docs.google.com/document/d/1XOxFtloiMuW24iqJ-zJnDzHl2KMxipTjJoxleJFz66A/edit?usp=sharing
JIRA: https://issues.apache.org/jira/browse/SPARK-31357
WIP PR: https://github.com/apache/spark/pull/28147

As part of a project to support common views across query engines like
Spark and Presto, my team used the view catalog API in Spark
implementation. The project has been in production over three months.

Thanks,
John Zhuge

Mime
View raw message