spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Rodriguez <ski.rodrig...@gmail.com>
Subject Re: Spark SQL Table Caching
Date Wed, 22 Jul 2015 22:59:19 GMT
I would be interested in the answer to this question, plus the relationship
between those and registerTempTable()

Pedro

On Tue, Jul 21, 2015 at 1:59 PM, Brandon White <bwwinthehouse@gmail.com>
wrote:

> A few questions about caching a table in Spark SQL.
>
> 1) Is there any difference between caching the dataframe and the table?
>
> df.cache() vs sqlContext.cacheTable("tableName")
>
> 2) Do you need to "warm up" the cache before seeing the performance
> benefits? Is the cache LRU? Do you need to run some queries on the table
> before it is cached in memory?
>
> 3) Is caching the table much faster than .saveAsTable? I am only seeing a
> 10 %- 20% performance increase.
>



-- 
Pedro Rodriguez
UCBerkeley 2014 | Computer Science
SnowGeek <http://SnowGeek.org>
pedro-rodriguez.com
ski.rodriguez@gmail.com
208-340-1703

Mime
View raw message