spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Earthson <>
Subject [Suggestion]Strange behavior for broadcast cleaning with spark 0.9
Date Fri, 09 May 2014 02:18:00 GMT
I'm using spark-0.9 with YARN.

Q: Why spark.cleaner.ttl setting could remove broadcast that still in use? I
think cleaner should not remove broadcasts still in the dependences of some
RDDs. It makes the value of spark.cleaner.ttl need to be set more carefully.

POINT: cleaner should not crash the task.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message