spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Herman Yu <>
Subject access spark thrift server from another spark session
Date Mon, 03 Oct 2016 21:10:31 GMT

I built spark data frame/dataset on top of several hive tables, and then registered dataframe/dataset
as temporary tables, as well as exposed the temporary table through spark thrift server. Now
the question is, this temporary table is only visible to the same spark session. How do I
make the registered temporary table available to another spark session? 

I have tried manually set the hive.server2.thrift.port in the new spark session,but the temporary
table is still not seen. I also tried connecting to thrift server through JDBC from the 2nd
spark session, it doesn’t work either (seems spark doesn’t like the idea accessing hive
through JDBC).

so the questions is: is it possible to share data frame/dataset based temporary tables through
Spark thrift server between multiple spark sessions?


To unsubscribe e-mail:

View raw message