spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (JIRA)" <>
Subject [jira] [Commented] (SPARK-17767) Spark SQL ExternalCatalog API custom implementation support
Date Mon, 03 Oct 2016 22:22:20 GMT


Dongjoon Hyun commented on SPARK-17767:

Thank you for confirming.

For Spark Thrift Server, I'm going to create another JIRA issue if the PR is merged.

The current PR is focusing on supporting generic ExternalCatalog in Spark side.

> Spark SQL ExternalCatalog API custom implementation support
> -----------------------------------------------------------
>                 Key: SPARK-17767
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.0.0
>            Reporter: Alex Liu
> There is no way/easy way to configure Spark to use customized ExternalCatalog.  Internal
source code is hardcoded to use either hive or in-memory metastore. Spark SQL thriftserver
is hardcoded to use HiveExternalCatalog. We should be able to create a custom external catalog
and thriftserver should be able to use it. Potentially Spark SQL thriftserver shouldn't depend
on Hive thriftserer.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message