spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr SmoliƄski <piotr.smolinski...@gmail.com>
Subject Re: Cached Parquet file paths problem
Date Tue, 19 Apr 2016 15:32:23 GMT
Solved it.

The anonymous RDDs can be cached in the cacheManager in SQLContext.

In order to remove all the cached content use:
      sqlContext.clearCache()

The warning symptom about failed data frame registration is the following
entry in the log:

16/04/16 20:18:39 [tp439928219-110]  WARN CacheManager: Asked to cache
already cached data.


Piotr

On Wed, Mar 30, 2016 at 3:10 PM, psmolinski <piotr.smolinski.77@gmail.com>
wrote:

> bumping up the topic.
>
> For the moment I stay with 1.5.2, but I would like to switch to 1.6.x and
> this issue is a blocker.
>
> Thanks,
> Piotr
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Cached-Parquet-file-paths-problem-tp26576p26637.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message