spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SRK <swethakasire...@gmail.com>
Subject Partitions cached by updatStateByKey does not seem to be getting evicted forever
Date Thu, 06 Jul 2017 16:59:46 GMT
Hi,

We use updateStateByKey in our Spark streaming application. The partitions
cached by updateStateByKey does not seem to be getting evicted. It was
getting evicted fine with spark.cleaner.ttl in 1.5.1. I am facing issues
with partitions not getting evicted with Stateful Streaming after Spark 2.1.
upgrade.

Any idea as to why this happens?


Thanks!



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Partitions-cached-by-updatStateByKey-does-not-seem-to-be-getting-evicted-forever-tp28827.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message