spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (Jira)" <j...@apache.org>
Subject [jira] [Reopened] (SPARK-31170) Spark Cli does not respect hive-site.xml and spark.sql.warehouse.dir
Date Thu, 19 Mar 2020 23:20:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-31170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dongjoon Hyun reopened SPARK-31170:
-----------------------------------

This is reverted because this broke all `hive-1.2` profile Jenkins jobs (2 SBT/2 Maven).

> Spark Cli does not respect hive-site.xml and spark.sql.warehouse.dir
> --------------------------------------------------------------------
>
>                 Key: SPARK-31170
>                 URL: https://issues.apache.org/jira/browse/SPARK-31170
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Kent Yao
>            Assignee: Kent Yao
>            Priority: Major
>             Fix For: 3.0.0
>
>
> In Spark CLI, we create a hive CliSessionState and it does not load the hive-site.xml.
So the configurations in hive-site.xml will not take effects like other spark-hive integration
apps.
> Also, the warehouse directory is not correctly picked. If the `default` database does
not exist, the CliSessionState will create one during the first time it talks to the metastore.
The `Location` of the default DB will be neither the value of spark.sql.warehousr.dir nor
the user-specified value of hive.metastore.warehourse.dir, but the default value of hive.metastore.warehourse.dir
which will always be `/user/hive/warehouse`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message