spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jungtaek Lim <kabhwan.opensou...@gmail.com>
Subject [DISCUSS] preferred behavior when fails to instantiate configured v2 session catalog
Date Fri, 23 Oct 2020 03:32:17 GMT
Hi devs,

I got another report regarding configuring v2 session catalog, when Spark
fails to instantiate the configured catalog. For now, it just simply logs
error message without exception information, and silently uses the default
session catalog.

https://github.com/apache/spark/blob/3819d39607392aa968595e3d97b84fedf83d08d9/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogManager.scala#L75-L95

IMO, as the user intentionally provides the session catalog, it shouldn't
fail back and just throw the exception. Otherwise (if we still want to do
the failback), we need to add the exception information in the error log
message at least.

Would like to hear the voices.

Thanks,
Jungtaek Lim (HeartSaVioR)

Mime
View raw message