spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurent Bastien Corbeil <>
Subject Temporary tables for Spark SQL
Date Tue, 12 Nov 2019 21:01:47 GMT

I am new to Spark, so I have a basic question which I couldn't find an
answer online.

If I want to run SQL queries on a Spark dataframe, do I have to create a
temporary table first?

I know I could use the Spark SQL API, but is there a way of simply reading
the data and run SQL queries without creating a temporary table?

If that is impossible, does that mean that running SQL queries is less
efficient than using the API since it does not require the use of temporary

Any help would be appreciated. Thanks,

View raw message