spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SRK <>
Subject Partitions cached by updatStateByKey does not seem to be getting evicted forever
Date Thu, 06 Jul 2017 16:59:46 GMT

We use updateStateByKey in our Spark streaming application. The partitions
cached by updateStateByKey does not seem to be getting evicted. It was
getting evicted fine with spark.cleaner.ttl in 1.5.1. I am facing issues
with partitions not getting evicted with Stateful Streaming after Spark 2.1.

Any idea as to why this happens?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message