spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Toth (Jira)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-29078) Spark shell fails if read permission is not granted to hive warehouse directory
Date Wed, 02 Oct 2019 19:26:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-29078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943095#comment-16943095
] 

Peter Toth commented on SPARK-29078:
------------------------------------

[~misutoth], if we look closer at the stacktrace ({{at org.apache.spark.sql.internal.SharedState.externalCatalog$lzycompute(SharedState.scala:114)}})
it shows that the AccessControlException issue is with the default database existence check
(on master branch this corresponds to https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala#L139:

{noformat}
    // Create default database if it doesn't exist
    if (!externalCatalog.databaseExists(SessionCatalog.DEFAULT_DATABASE)) {
      // There may be another Spark application creating default database at the same time,
here we
{noformat}

This exception is expected if the user doesn't have access to the directory of the default
database. In that case the user can't use Spark SQL.

I would suggest closing this ticket.

> Spark shell fails if read permission is not granted to hive warehouse directory
> -------------------------------------------------------------------------------
>
>                 Key: SPARK-29078
>                 URL: https://issues.apache.org/jira/browse/SPARK-29078
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Mihaly Toth
>            Priority: Major
>
> Similarly to SPARK-20256, in {{SharedSessionState}} when {{GlobalTempViewManager}} is
created, it is checked that there is no database exists that has the same name as of the global
temp database (name is configurable with {{spark.sql.globalTempDatabase}}) , because that
is a special database, which should not exist in the metastore. For this, a read permission
is required on the warehouse directory at the moment, which on the other hand would allow
listing all the databases of all users.
> When such a read access is not granted for security reasons, an access violation exception
should be ignored upon such initial validation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message