spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Toth (Jira)" <>
Subject [jira] [Commented] (SPARK-29078) Spark shell fails if read permission is not granted to hive warehouse directory
Date Wed, 02 Oct 2019 19:26:00 GMT


Peter Toth commented on SPARK-29078:

[~misutoth], if we look closer at the stacktrace ({{at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)}})
it shows that the AccessControlException issue is with the default database existence check
(on master branch this corresponds to

    // Create default database if it doesn't exist
    if (!externalCatalog.databaseExists(SessionCatalog.DEFAULT_DATABASE)) {
      // There may be another Spark application creating default database at the same time,
here we

This exception is expected if the user doesn't have access to the directory of the default
database. In that case the user can't use Spark SQL.

I would suggest closing this ticket.

> Spark shell fails if read permission is not granted to hive warehouse directory
> -------------------------------------------------------------------------------
>                 Key: SPARK-29078
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Mihaly Toth
>            Priority: Major
> Similarly to SPARK-20256, in {{SharedSessionState}} when {{GlobalTempViewManager}} is
created, it is checked that there is no database exists that has the same name as of the global
temp database (name is configurable with {{spark.sql.globalTempDatabase}}) , because that
is a special database, which should not exist in the metastore. For this, a read permission
is required on the warehouse directory at the moment, which on the other hand would allow
listing all the databases of all users.
> When such a read access is not granted for security reasons, an access violation exception
should be ignored upon such initial validation.

This message was sent by Atlassian Jira

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message