spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Earthson <Earthson...@gmail.com>
Subject [Suggestion]Strange behavior for broadcast cleaning with spark 0.9
Date Fri, 09 May 2014 02:18:00 GMT
I'm using spark-0.9 with YARN.

Q: Why spark.cleaner.ttl setting could remove broadcast that still in use? I
think cleaner should not remove broadcasts still in the dependences of some
RDDs. It makes the value of spark.cleaner.ttl need to be set more carefully.

POINT: cleaner should not crash the task.





--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Suggestion-Strange-behavior-for-broadcast-cleaning-with-spark-0-9-tp5512.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message