Solved it.

The anonymous RDDs can be cached in the cacheManager in SQLContext.

In order to remove all the cached content use:
      sqlContext.clearCache()

The warning symptom about failed data frame registration is the following entry in the log:

16/04/16 20:18:39 [tp439928219-110]  WARN CacheManager: Asked to cache already cached data.


Piotr

On Wed, Mar 30, 2016 at 3:10 PM, psmolinski <piotr.smolinski.77@gmail.com> wrote:
bumping up the topic.

For the moment I stay with 1.5.2, but I would like to switch to 1.6.x and
this issue is a blocker.

Thanks,
Piotr



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Cached-Parquet-file-paths-problem-tp26576p26637.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org