spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Earthson <Earthson...@gmail.com>
Subject cache not work as expected for iteration?
Date Sun, 04 May 2014 03:16:49 GMT
I'm using spark for LDA impementation. I need cache RDD for next step of
Gibbs Sampling, and cached the result and the cache previous could be
uncache. Something like LRU cache should delete the previous cache because
it is never used then, but the cache runs into confusion:

Here is the code:)
<https://github.com/Earthson/sparklda/blob/master/src/main/scala/net/earthson/nlp/lda/lda.scala#L99>
 

<http://apache-spark-user-list.1001560.n3.nabble.com/file/n5292/sparklda_cache1.png>


<http://apache-spark-user-list.1001560.n3.nabble.com/file/n5292/sparklda_cache2.png>






--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/cache-not-work-as-expected-for-iteration-tp5292.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message