spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From epettijohn <>
Subject Re: How to control saveAsTable() warehouse path?
Date Thu, 15 Dec 2016 22:57:47 GMT
I don't profess to be an expert on this, but I did face the same problem.  A
couple of possibilities:

1.  If your default Hive database is stored in "/tmp/hive/warehouse/...",
then that could be the issue.  I recommend creating a database on s3a and
then storing the table there ( .saveAsTable('s3db.test_table',... )

2.  Have you tried setting spark.sql.warehouse.dir?  I know the docs say
that it uses the hive metastore dir first, but I've had some luck with
setting that one.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message